图像

敏捷测试

Agile Testing

测试人员和敏捷团队实用指南

A Practical Guide for Testers and Agile Teams

丽莎克里斯平珍

妮特格雷戈里

Lisa Crispin

Janet Gregory

图像

新泽西州上萨德尔河 • 波士顿 • 印第安纳波利斯 • 旧金山

纽约 • 多伦多 • 蒙特利尔 • 伦敦 • 慕尼黑 • 巴黎 • 马德里

开普敦 • 悉尼 • 东京 • 新加坡 • 墨西哥城



Upper Saddle River, NJ • Boston • Indianapolis • San Francisco

New York • Toronto • Montreal • London • Munich • Paris • Madrid

Capetown • Sydney • Tokyo • Singapore • Mexico City

奉献精神

Dedication

致我的丈夫鲍勃唐宁——你是蜜蜂的膝盖!

——丽莎

To my husband, Bob Downing—you’re the bee’s knees!

—Lisa

致 Jack、Dana 和 Susan,以及我家族中的所有作家。

——珍妮特

To Jack, Dana, and Susan, and to all the writers in my family.

—Janet

还有我们最喜欢的驴和龙。

——丽莎和珍妮特

And to all our favorite donkeys and dragons.

—Lisa and Janet

目录

Table of Contents

迈克·科恩的前言

Foreword by Mike Cohn

布赖恩·马里克的前言

Foreword by Brian Marick

前言

Preface

致谢

Acknowledgments

关于作者

About the Authors

第一部分 引言

Part I. Introduction

第 1 章什么是敏捷测试?

Chapter 1. What Is Agile Testing, Anyway?

敏捷价值观

Agile Values

“敏捷测试”是什么意思?

What Do We Mean by “Agile Testing”?

敏捷团队角色和活动的一点背景

A Little Context for Roles and Activities on an Agile Team

客户团队

Customer Team

开发团队

Developer Team

客户和开发团队之间的互动

Interaction between Customer and Developer Teams

敏捷测试有何不同?

How Is Agile Testing Different?

在传统团队中工作

Working on Traditional Teams

在敏捷团队中工作

Working on Agile Teams

传统与敏捷测试

Traditional vs. Agile Testing

全团队方法

Whole-Team Approach

概括

Summary

第 2 章敏捷测试人员的十项原则

Chapter 2. Ten Principles for Agile Testers

什么是敏捷测试员?

What’s an Agile Tester?

敏捷测试思维模式

The Agile Testing Mind-Set

应用敏捷原则和价值观

Applying Agile Principles and Values

提供持续反馈

Provide Continuous Feedback

为客户提供价值

Deliver Value to the Customer

实现面对面交流

Enable Face-to-Face Communication

有勇气

Have Courage

把事情简单化

Keep It Simple

实践持续改进

Practice Continuous Improvement

应对变化

Respond to Change

自组织

Self-Organize

以人为本

Focus on People

享受

Enjoy

增加价值

Adding Value

概括

Summary

第二部分。组织挑战

Part II. Organizational Challenges

第 3 章文化挑战

Chapter 3. Cultural Challenges

组织文化

Organizational Culture

品质理念

Quality Philosophy

可持续步伐

Sustainable Pace

客户关系

Customer Relationships

组织规模

Organization Size

授权你的团队

Empower Your Team

测试/QA 团队成功采用敏捷的障碍

Barriers to Successful Agile Adoption by Test/QA Teams

失去身份

Loss of Identity

其他角色

Additional Roles

缺乏培训

Lack of Training

不理解敏捷概念

Not Understanding Agile Concepts

过去的经验/态度

Past Experience/Attitude

角色之间的文化差异

Cultural Differences among Roles

引入变革

Introducing Change

谈论恐惧

Talk about Fears

赋予团队所有权

Give Team Ownership

庆祝成功

Celebrate Success

管理层期望

Management Expectations

管理者的文化变革

Cultural Changes for Managers

说经理的语言

Speaking the Manager’s Language

改变来之不易

Change Doesn’t Come Easy

要有耐心

Be Patient

让他们感到痛苦

Let Them Feel Pain

建立你的信誉

Build Your Credibility

致力于你自己的职业发展

Work On Your Own Professional Development

当心素质警察的心态

Beware the Quality Police Mentality

用脚投票

Vote with Your Feet

概括

Summary

第 4 章团队后勤

Chapter 4. Team Logistics

团队架构

Team Structure

独立的质量保证团队

Independent QA Teams

将测试人员集成到敏捷项目中

Integration of Testers into an Agile Project

敏捷项目团队

Agile Project Teams

实体物流

Physical Logistics

资源

Resources

测试人员-开发人员比率

Tester-Developer Ratio

雇用敏捷测试人员

Hiring an Agile Tester

建立团队

Building a Team

自组织团队

Self-Organizing Team

涉及其他团队

Involving Other Teams

每个团队成员都具有同等价值

Every Team Member Has Equal Value

绩效和奖励

Performance and Rewards

你能做什么?

What Can You Do?

概括

Summary

第 5 章过渡典型流程

Chapter 5. Transitioning Typical Processes

寻求轻量级流程

Seeking Lightweight Processes

指标

Metrics

精益测量

Lean Measurements

为什么我们需要指标

Why We Need Metrics

不应该用指标做什么

What Not to Do with Metrics

交流指标

Communicating Metrics

指标投资回报率

Metrics ROI

缺陷追踪

Defect Tracking

为什么我们应该使用缺陷跟踪系统 (DTS)?

Why Should We Use a Defect Tracking System (DTS)?

为什么我们不应该使用 DTS?

Why Shouldn’t We Use a DTS?

缺陷跟踪工具

Defect Tracking Tools

保持专注

Keep Your Focus

测试计划

Test Planning

测试策略与测试计划

Test Strategy vs. Test Planning

可追溯性

Traceability

现有流程和模型

Existing Processes and Models

审计

Audits

框架、模型和标准

Frameworks, Models, and Standards

概括

Summary

第三部分。敏捷测试象限

Part III. The Agile Testing Quadrants

第 6 章测试的目的

Chapter 6. The Purpose of Testing

敏捷测试象限

The Agile Testing Quadrants

支持团队的测试

Tests that Support the Team

批评产品的测试

Tests that Critique the Product

知道故事何时完成

Knowing When a Story Is Done

共同责任

Shared Responsibility

管理技术债务

Managing Technical Debt

在上下文中测试

Testing in Context

概括

Summary

第 7 章支持团队的面向技术的测试

Chapter 7. Technology-Facing Tests that Support the Team

敏捷测试基础

An Agile Testing Foundation

象限 1 测试的目的

The Purpose of Quadrant 1 Tests

配套基础设施

Supporting Infrastructure

为什么要编写和执行这些测试?

Why Write and Execute These Tests?

让我们走得更快,做得更多

Lets Us Go Faster and Do More

使测试人员的工作更轻松

Making Testers’ Jobs Easier

设计时考虑测试

Designing with Testing in Mind

及时反馈

Timely Feedback

面向技术的测试在哪里停止?

Where Do Technology-Facing Tests Stop?

如果团队不做这些测试怎么办?

What If the Team Doesn’t Do These Tests?

测试人员可以做什么?

What Can Testers Do?

管理者能做什么?

What Can Managers Do?

这是一个团队问题

It’s a Team Problem

工具包

Toolkit

源代码控制

Source Code Control

集成开发环境

IDEs

构建工具

Build Tools

构建自动化工具

Build Automation Tools

单元测试工具

Unit Test Tools

概括

Summary

第 8 章支持团队的面向业务的测试

Chapter 8. Business-Facing Tests that Support the Team

通过面向业务的测试推动开发

Driving Development with Business-Facing Tests

需求困境

The Requirements Quandary

通用语言

Common Language

引出需求

Eliciting Requirements

提前澄清

Advance Clarity

满意的条件

Conditions of Satisfaction

涟漪效应

Ripple Effects

薄片,小块

Thin Slices, Small Chunks

我们怎么知道我们已经完成了?

How Do We Know We’re Done?

测试降低风险

Tests Mitigate Risk

可测试性和自动化

Testability and Automation

概括

Summary

第 9 章支持团队的面向业务的测试工具包

Chapter 9. Toolkit for Business-Facing Tests that Support the Team

面向业务的测试工具策略

Business-Facing Test Tool Strategy

引出示例和需求的工具

Tools to Elicit Examples and Requirements

清单

Checklists

思维导图

Mind Maps

电子表格

Spreadsheets

模型

Mock-Ups

流程图

Flow Diagrams

基于软件的工具

Software-Based Tools

基于示例的自动化测试工具

Tools for Automating Tests Based on Examples

在 GUI 和 API 级别下测试的工具

Tools to Test below the GUI and API Level

通过 GUI 进行测试的工具

Tools for Testing through the GUI

编写测试的策略

Strategies for Writing Tests

增量构建测试

Build Tests Incrementally

保持测试通过

Keep the Tests Passing

使用适当的测试设计模式

Use Appropriate Test Design Patterns

关键字和数据驱动测试

Keyword and Data-Driven Tests

可测试性

Testability

代码设计和测试设计

Code Design and Test Design

自动化与手动象限 2 测试

Automated vs. Manual Quadrant 2 Tests

测试管理

Test Management

概括

Summary

第 10 章评价产品的面向业务的测试

Chapter 10. Business-Facing Tests that Critique the Product

第三象限简介

Introduction to Quadrant 3

示范

Demonstrations

场景测试

Scenario Testing

探索性测试

Exploratory Testing

基于会话的测试

Session-Based Testing

自动化和探索性测试

Automation and Exploratory Testing

探索性测试仪

An Exploratory Tester

可用性测试

Usability Testing

用户需求和角色测试

User Needs and Persona Testing

导航

Navigation

查看比赛

Check Out the Competition

图形用户界面的背后

Behind the GUI

API测试

API Testing

网页服务

Web Services

测试文档和文档

Testing Documents and Documentation

用户文档

User Documentation

报告

Reports

辅助探索性测试的工具

Tools to Assist with Exploratory Testing

测试设置

Test Setup

测试数据生成

Test Data Generation

监控工具

Monitoring Tools

模拟器

Simulators

模拟器

Emulators

概括

Summary

第 11 章使用面向技术的测试来评价产品

Chapter 11. Critiquing the Product Using Technology-Facing Tests

象限 4 简介

Introduction to Quadrant 4

谁做的?

Who Does It?

你什么时候做?

When Do You Do It?

“能力”测试

“ility” Testing

安全

Security

可维护性

Maintainability

互操作性

Interoperability

兼容性

Compatibility

可靠性

Reliability

安装性

Installability

“能力”总结

“ility” Summary

性能、负载、压力和可扩展性测试

Performance, Load, Stress, and Scalability Testing

可扩展性

Scalability

性能和负载测试

Performance and Load Testing

性能和负载测试工具

Performance and Load-Testing Tools

基线

Baseline

测试环境

Test Environments

内存管理

Memory Management

概括

Summary

第 12 章测试象限总结

Chapter 12. Summary of Testing Quadrants

测试象限审查

Review of the Testing Quadrants

系统测试示例

A System Test Example

应用程序

The Application

团队和流程

The Team and the Process

测试驱动开发

Tests Driving Development

单元测试

Unit Tests

验收测试

Acceptance Tests

自动化

Automation

自动化功能测试结构

The Automated Functional Test Structure

网页服务

Web Services

嵌入式测试

Embedded Testing

通过面向业务的测试来评价产品

Critiquing the Product with Business-Facing Tests

探索性测试

Exploratory Testing

测试数据源

Testing Data Feeds

端到端测试

The End-to-End Tests

用户验收测试

User Acceptance Testing

可靠性

Reliability

文档

Documentation

记录测试代码

Documenting the Test Code

报告测试结果

Reporting the Test Results

使用敏捷测试象限

Using the Agile Testing Quadrants

概括

Summary

第四部分。自动化

Part IV. Automation

第 13 章 为什么我们想要自动化测试以及是什么阻碍了我们

Chapter 13. Why We Want to Automate Tests and What Holds Us Back

为什么要自动化?

Why Automate?

手动测试耗时过长

Manual Testing Takes Too Long

手动流程容易出错

Manual Processes Are Error Prone

自动化使人们能够尽最大努力工作

Automation Frees People to Do Their Best Work

自动回归测试提供安全网

Automated Regression Tests Provide a Safety Net

自动化测试会尽早且经常提供反馈

Automated Tests Give Feedback, Early and Often

驱动编码可以做更多的测试和示例

Tests and Examples that Drive Coding Can Do More

测试是很棒的文档

Tests Are Great Documentation

投资回报率和回报

ROI and Payback

自动化的障碍——阻碍的事情

Barriers to Automation—Things that Get in the Way

布雷特的名单

Bret’s List

我们的名单

Our List

程序员的态度——“为什么要自动化?”

Programmers’ Attitude—“Why Automate?”

“痛苦的驼峰”(学习曲线)

The “Hump of Pain” (The Learning Curve)

初始投资

Initial Investment

始终在变化中的代码

Code that’s Always in Flux

遗留代码

Legacy Code

害怕

Fear

旧习惯

Old Habits

我们能克服这些障碍吗?

Can We Overcome These Barriers?

概括

Summary

第 14 章敏捷测试自动化策略

Chapter 14. An Agile Test Automation Strategy

测试自动化的敏捷方法

An Agile Approach to Test Automation

自动化测试类别

Automation Test Categories

测试自动化金字塔

Test Automation Pyramid

我们可以自动化什么?

What Can We Automate?

持续集成、构建和部署

Continuous Integration, Builds, and Deploys

单元和组件测试

Unit and Component Tests

API 或网络服务测试

API or Web Services Testing

GUI 背后的测试

Testing behind the GUI

测试图形用户界面

Testing the GUI

负载测试

Load Tests

比较

Comparisons

重复性任务

Repetitive Tasks

数据创建或设置

Data Creation or Setup

我们不应该自动化什么?

What Shouldn’t We Automate?

可用性测试

Usability Testing

探索性测试

Exploratory Testing

永远不会失败的测试

Tests that Will Never Fail

一次性测试

One-Off Tests

什么可能难以自动化?

What Might Be Hard to Automate?

制定自动化战略——我们从哪里开始?

Developing an Automation Strategy—Where Do We Start?

哪里最痛?

Where Does It Hurt the Most?

多层方法

Multi-Layered Approach

考虑测试设计和维护

Think about Test Design and Maintenance

选择合适的工具

Choosing the Right Tools

将敏捷原则应用于测试自动化

Applying Agile Principles to Test Automation

把事情简单化

Keep It Simple

迭代反馈

Iterative Feedback

全团队方法

Whole-Team Approach

花时间做正确的事

Taking the Time to Do It Right

通过实践学习

Learn by Doing

将敏捷编码实践应用于测试

Apply Agile Coding Practices to Tests

为测试提供数据

Supplying Data for Tests

数据生成工具

Data Generation Tools

避免数据库访问

Avoid Database Access

当数据库访问不可避免甚至需要时

When Database Access Is Unavoidable or Even Desirable

了解您的需求

Understand Your Needs

评估自动化工具

Evaluating Automation Tools

确定自动化工具的要求

Identifying Requirements for Your Automation Tool

一次一个工具

One Tool at a Time

选择工具

Choosing Tools

敏捷友好的工具

Agile-Friendly Tools

实施自动化

Implementing Automation

管理自动化测试

Managing Automated Tests

组织测试

Organizing Tests

组织测试结果

Organizing Test Results

开始吧

Go Get Started

概括

Summary

第五部分测试人员生命中的一次迭代

Part V. An Iteration in the Life of a Tester

第 15 章发布或主题规划中的测试人员活动

Chapter 15. Tester Activities in Release or Theme Planning

发布计划的目的

The Purpose of Release Planning

浆纱

Sizing

如何调整故事大小

How to Size Stories

测试人员在调整故事中的作用

The Tester’s Role in Sizing Stories

规模故事的一个例子

An Example of Sizing Stories

优先排序

Prioritizing

为什么我们优先考虑故事

Why We Prioritize Stories

确定优先级时的测试注意事项

Testing Considerations While Prioritizing

范围内有什么?

What’s in Scope?

截止日期和时间表

Deadlines and Timelines

关注价值

Focus on Value

全系统影响

System-Wide Impact

第三方参与

Third-Party Involvement

测试计划

Test Planning

从哪儿开始

Where to Start

为什么要编写测试计划?

Why Write a Test Plan?

测试类型

Types of Testing

基础设施

Infrastructure

测试环境

Test Environments

测试数据

Test Data

检测结果

Test Results

测试计划备选方案

Test Plan Alternatives

轻量级测试计划

Lightweight Test Plans

使用测试矩阵

Using a Test Matrix

测试电子表格

Test Spreadsheet

一块白板

A Whiteboard

自动化测试列表

Automated Test List

为可见性做准备

Preparing for Visibility

跟踪测试任务和状态

Tracking Test Tasks and Status

交流测试结果

Communicating Test Results

发布指标

Release Metrics

概括

Summary

第 16 章开始运行

Chapter 16. Hit the Ground Running

主动

Be Proactive

好处

Benefits

你真的需要这个吗?

Do You Really Need This?

提前准备的潜在缺点

Potential Downsides to Advance Preparation

提前澄清

Advance Clarity

客户用一种声音说话

Customers Speak with One Voice

故事大小

Story Size

地域分散的团队

Geographically Dispersed Teams

例子

Examples

测试策略

Test Strategies

确定缺陷的优先​​级

Prioritize Defects

资源

Resources

概括

Summary

第 17 章迭代启动

Chapter 17. Iteration Kickoff

迭代计划

Iteration Planning

学习细节

Learning the Details

考虑所有观点

Considering All Viewpoints

编写任务卡

Writing Task Cards

决定工作量

Deciding on Workload

可测试的故事

Testable Stories

与客户合作

Collaborate with Customers

高级测试和示例

High-Level Tests and Examples

与客户一起审查

Reviewing with Customers

与程序员一起审查

Reviewing with Programmers

测试用例作为文档

Test Cases as Documentation

概括

Summary

第 18 章编码和测试

Chapter 18. Coding and Testing

推动发展

Driving Development

从简单开始

Start Simple

增加复杂性

Add Complexity

评估风险

Assess Risk

编码和测试共同进步

Coding and Testing Progress Together

识别变化

Identify Variations

三的力量

Power of Three

专注于一个故事

Focus on One Story

批评产品的测试

Tests that Critique the Product

与程序员合作

Collaborate with Programmers

结对测试

Pair Testing

“给我看看”

“Show Me”

与客户交谈

Talk to Customers

显示客户

Show Customers

了解业务

Understand the Business

完成测试任务

Completing Testing Tasks

处理错误

Dealing with Bugs

它是缺陷还是特性?

Is It a Defect or Is It a Feature?

技术债务

Technical Debt

零错误容忍

Zero Bug Tolerance

一切都与选择有关

It’s All about Choices

决定记录哪些错误

Decide Which Bugs to Log

选择何时修复错误

Choose When to Fix Your Bugs

选择您应该用来记录错误的媒体

Choose the Media You Should Use to Log a Bug

处理错误的替代方案和建议

Alternatives and Suggestions for Dealing with Bugs

从简单开始

Start Simple

促进沟通

Facilitate Communication

测试人员促进沟通

Testers Facilitate Communication

分布式团队

Distributed Teams

回归测试

Regression Tests

保持构建“绿色”

Keep the Build “Green”

保持快速构建

Keep the Build Quick

构建回归套件

Building a Regression Suite

检查“大局”

Checking the “Big Picture”

资源

Resources

迭代指标

Iteration Metrics

衡量进展

Measuring Progress

缺陷指标

Defect Metrics

概括

Summary

第 19 章结束迭代

Chapter 19. Wrap Up the Iteration

迭代演示

Iteration Demo

回顾

Retrospectives

开始、停止、继续

Start, Stop, Continue

改进的想法

Ideas for Improvements

庆祝成功

Celebrate Successes

概括

Summary

第 20 章 成功交付

Chapter 20. Successful Delivery

是什么造就了产品?

What Makes a Product?

计划足够的时间进行测试

Planning Enough Time for Testing

结束游戏

The End Game

测试发布候选

Testing the Release Candidate

在暂存环境中测试

Test on a Staging Environment

最终非功能测试

Final Nonfunctional Testing

与外部应用程序集成

Integration with External Applications

数据转换和数据库更新

Data Conversion and Database Updates

安装测试

Installation Testing

沟通

Communication

如果还没有准备好怎么办?

What If It’s Not Ready?

客户测试

Customer Testing

UAT

UAT

阿尔法/贝塔测试

Alpha/Beta Testing

开发后测试周期

Post-Development Testing Cycles

可交付成果

Deliverables

发布产品

Releasing the Product

发布验收标准

Release Acceptance Criteria

发布管理

Release Management

包装

Packaging

生产支持

Production Support

了解对业务的影响

Understand Impact to Business

客户期望

Customer Expectations

概括

Summary

第六部分。概括

Part VI. Summary

第 21 章关键成功因素

Chapter 21. Key Success Factors

成功因素 1:使用整个团队的方法

Success Factor 1: Use the Whole-Team Approach

成功因素 2:采用敏捷测试思维模式

Success Factor 2: Adopt an Agile Testing Mind-Set

成功因素 3:自动化回归测试

Success Factor 3: Automate Regression Testing

成功因素 4:提供和获取反馈

Success Factor 4: Provide and Obtain Feedback

成功因素 5:建立核心实践基础

Success Factor 5: Build a Foundation of Core Practices

持续集成

Continuous Integration

测试环境

Test Environments

管理技术债务

Manage Technical Debt

增量工作

Working Incrementally

编码和测试是一个过程的一部分

Coding and Testing Are Part of One Process

实践之间的协同作用

Synergy between Practices

成功因素 6:与客户合作

Success Factor 6: Collaborate with Customers

成功因素 7:放眼大局

Success Factor 7: Look at the Big Picture

概括

Summary

词汇表

Glossary

参考书目

Bibliography

指数

Index

赞扬敏捷测试

Praise for Agile Testing

“随着敏捷方法进入主流,我们已经了解了很多关于测试规程如何适应敏捷项目的知识。Lisa 和 Janet 让我们清楚地了解在敏捷测试中应该做什么,应该避免什么。”

“As Agile methods have entered the mainstream, we’ve learned a lot about how the testing discipline fits into Agile projects. Lisa and Janet give us a solid look at what to do, and what to avoid, in Agile testing.”

—Ron Jeffries,www.XProgramming.com

—Ron Jeffries, www.XProgramming.com

“对敏捷及其如何影响软件测试社区的精彩介绍!”

“An excellent introduction to agile and how it affects the software test community!”

—Gerard Meszaros,敏捷指导和精益软件开发咨询公司 Solution Frameworks, Inc. 的敏捷实践主管和首席测试策略师

—Gerard Meszaros, Agile Practice Lead and Chief Test Strategist at Solution Frameworks, Inc., an agile coaching and lean software development consultancy

“在体育和音乐领域,人们知道练习技巧的重要性,直到它成为他们做事方式的一部分。本书介绍了软件开发中一些最基本的技术——如何将质量构建到代码中——这些技术应该成为每个开发团队的第二天性。这本书广泛而深入地介绍了如何将测试移至开发过程的前端,并大量介绍了使本书栩栩如生的现实示例。”

“In sports and music, people know the importance of practicing technique until it becomes a part of the way they do things. This book is about some of the most fundamental techniques in software development—how to build quality into code—techniques that should become second nature to every development team. The book provides both broad and in-depth coverage of how to move testing to the front of the development process, along with a liberal sprinkling of real-life examples that bring the book to life.”

—Mary Poppendieck,《精益软件开发》《实施精益软件开发》一书的作者

—Mary Poppendieck, Author of Lean Software Development and Implementing Lean Software Development

“令人耳目一新的实用主义。充满智慧。没有教条。这本书改变了游戏规则。每个软件专业人士都应该阅读它。”

“Refreshingly pragmatic. Chock-full of wisdom. Absent of dogma. This book is a game-changer. Every software professional should read it.”

——Bob Martin 叔叔,Object Mentor, Inc.

—Uncle Bob Martin, Object Mentor, Inc.

“通过敏捷测试,Lisa 和 Janet 使用他们对测试的整体敏感性来描述愿意提高测试效率的测试人员和团队的文化转变。现实生活中的项目经验和特定技术的结合提供了一种学习和适应不断变化的项目需求的绝佳方式。”

“With Agile Testing, Lisa and Janet have used their holistic sensibility of testing to describe a culture shift for testers and teams willing to elevate their test effectiveness. The combination of real-life project experiences and specific techniques provide an excellent way to learn and adapt to continually changing project needs.”

—Adam Geras,理学硕士 开发测试人员,Ideaca 知识服务

—Adam Geras, M.Sc. Developer-Tester, Ideaca Knowledge Services

“在敏捷项目中,每个人似乎都在问,'但是,测试呢?' 这完全是开发团队、测试团队的责任,还是开发人员和测试人员之间的协作?或者,“我们应该自动化多少测试?” 丽莎和珍妮特写了一本书,终于回答了这些类型的问题,甚至更多!无论您是测试人员、开发人员还是经理,您都将从他们在这本出色的书中分享的真实工作经验中学到许多很好的例子和故事。”

“On Agile projects, everyone seems to ask, ‘But, what about testing?’ Is it the development team’s responsibility entirely, the testing team, or a collaborative effort between developers and testers? Or, ‘How much testing should we automate?’ Lisa and Janet have written a book that finally answers these types of questions and more! Whether you’re a tester, developer, or manager, you’ll learn many great examples and stories from the real-world work experiences they’ve shared in this excellent book.”

—Paul Duvall,Stelligent 的首席技术官和《持续集成:提高软件质量和降低风险》的合著者

—Paul Duvall, CTO of Stelligent and co-author of Continuous Integration: Improving Software Quality and Reducing Risk

“终于出版了一本面向敏捷团队测试人员的书,它承认不只有一种正确的方法!敏捷测试全面覆盖了测试人员在转向敏捷时面临的问题:从工具和指标到角色和流程。它以来自许多贡献者的大量故事和示例进行了说明,清楚地描述了当今成功的敏捷测试人员正在做什么。”

“Finally a book for testers on Agile teams that acknowledges there is not just one right way! Agile Testing provides comprehensive coverage of the issues testers face when they move to Agile: from tools and metrics to roles and process. Illustrated with numerous stories and examples from many contributors, it gives a clear picture of what successful Agile testers are doing today.”

—WatirCraft 首席技术官兼 Watir 首席开发人员 Bret Pettichord

—Bret Pettichord, Chief Technical Officer of WatirCraft and Lead Developer of Watir

迈克·科恩的前言

Foreword by Mike Cohn

“质量是内在的,”程序员不断告诉我。作为拟议收购的一部分,我的老板要求我对开发团队及其产品进行最后的尽职调查。我们已经确定该公司最近推出的产品在市场上表现良好,但我要确保我们不会买到比收益更多的麻烦。所以我把时间都花在了开发团队身上。我一直在寻找可能因匆忙发布产品而引起的问题。我想知道,“代码干净吗?是否存在只能由一名开发人员处理的模块?是否有成百上千的缺陷等待被发现?” 当我问及团队的测试方法时,“质量是内在的”是我得到的答案。

“Quality is baked in,” the programmers kept telling me. As part of a proposed acquisition, my boss had asked me to perform some final due diligence on the development team and its product. We’d already established that the company’s recently launched product was doing well in the market, but I was to make sure we were not about to buy more trouble than benefit. So I spent my time with the development team. I was looking for problems that might arise from having rushed the product into release. I wondered, “Was the code clean? Were there modules that could only be worked on by one developer? Were there hundreds or thousands of defects waiting to be discovered?” And when I asked about the team’s approach to testing, “Quality is baked in” was the answer I got.

因为这种相当不寻常的口语化可能意味着任何事情,所以我进一步追问。我发现这是公司创始人表达质量先驱 W. Edwards Deming 著名的十四点之一的速记:将质量融入产品而不是在以后尝试测试它。

Because this rather unusual colloquialism could have meant just about anything, I pressed further. What I found was that this was the company founder’s shorthand for expressing one of quality pioneer W. Edwards Deming’s famous fourteen points: Build quality into the product rather than trying to test it in later.

将质量构建到他们的产品中的想法是敏捷团队工作方式的核心。敏捷团队在短期迭代中工作,部分是为了确保应用程序保持已知的质量状态。敏捷团队是高度跨职能的,程序员、测试人员和其他人在每次迭代中并肩工作,这样质量就可以通过验收测试驱动开发、高度重视自动化测试和整体-团队思维。优秀的敏捷团队通过不断构建他们的产品,在新工作完成后几分钟内整合新工作,从而融入质量。敏捷团队利用重构等技术和对简单性的偏好来防止技术债务的积累。

The idea of building quality into their products is at the heart of how agile teams work. Agile teams work in short iterations in part to ensure that the application remains at a known state of quality. Agile teams are highly cross-functional, with programmers, testers, and others working side by side throughout each iteration so that quality can be baked into products through techniques such as acceptance-test driven development, a heavy emphasis on automated testing, and whole-team thinking. Good agile teams bake quality in by building their products continuously, integrating new work within minutes of its being completed. Agile teams utilize techniques such as refactoring and a preference for simplicity in order to prevent technical debt from accumulating.

学习如何做这些事情是很困难的,对于测试人员来说尤其如此,他们的角色在以前的书中很少受到关注。幸运的是,您现在手里拿着的这本书回答了每个开始从事敏捷项目的测试人员心中的问题,例如:

Learning how to do these things is difficult, and especially so for testers, whose role has been given scant attention in previous books. Fortunately, the book you now hold in your hands answers questions on the mind of every tester who’s beginning to work on an agile project, such as:

图像我的角色和职责是什么?

What are my roles and responsibilities?

图像我如何与程序员更紧密地合作?

How do I work more closely with programmers?

图像我们自动化了多少,我们如何开始自动化?

How much do we automate, and how do we start automating?

丽莎和珍妮特的经历在书中的每一页都闪闪发光。然而,这本书不仅仅是他们的故事。在本书中,他们结合了来自真实世界的敏捷测试人员的数十个故事。这些故事构成了这本书的核心,也是它如此独特的原因。在象牙塔中大喊“这是进行敏捷测试的方法”是一回事。讲述团队的故事是另一回事,这些团队在可用性测试、抵制自动化的遗留代码、习惯于传统阶段门开发的过渡测试人员、“跟上”短迭代的测试等挑战中变得敏捷并取得胜利,并知道某个功能何时“完成”。

The experience of Lisa and Janet shines through on every page of the book. However, this book is not just their story. Within this book, they incorporate dozens of stories from real-world agile testers. These stories form the heart of the book and are what makes it so unique. It’s one thing to shout from the ivory tower, “Here’s how to do agile testing.” It’s another to tell the stories of the teams that have struggled and then emerged agile and victorious over challenges such as usability testing, legacy code that resists automation, transitioning testers used to traditional phase-gate development, testing that “keeps up” with short iterations, and knowing when a feature is “done.”

Lisa 和 Janet 一开始就在那里,学习如何进行敏捷测试,当时普遍认为敏捷团队不需要测试人员,程序员可以自己提高质量。多年来,通过文章、会议演示以及与他们的客户和团队的合作,Lisa 和 Janet 帮助我们看到了测试人员在敏捷项目中扮演的丰富角色。在这本书中,Lisa 和 Janet 使用测试自动化金字塔、Brian Marick(他自己是另一位世界级的敏捷测试员)的敏捷测试象限,以及其他技术来展示在认为测试是必要的但不可行的心态中缺失了多少测试人员不是。

Lisa and Janet were there at the beginning, learning how to do agile testing back when the prevailing wisdom was that agile teams didn’t need testers and that programmers could bake quality in by themselves. Over the years and through articles, conference presentations, and working with their clients and teams, Lisa and Janet have helped us see the rich role to be filled by testers on agile projects. In this book, Lisa and Janet use a test automation pyramid, the agile testing quadrants of Brian Marick (himself another world-class agile tester), and other techniques to show how much was missing from a mind-set that said testing is necessary but testers aren’t.

如果你想学习如何将质量融入你的产品,或者是一个有抱负的敏捷测试人员,想了解你的角色,我认为没有比 Lisa 和 Janet 更好的指南了。

If you want to learn how to bake quality into your products or are an aspiring agile tester seeking to understand your role, I can think of no better guides than Lisa and Janet.

布赖恩·马里克的前言

Foreword by Brian Marick

想象一下,你自己掠过几千年前的风景,看着下面的人。他们在充满敌意的领土上勉强维持生计,打猎、捕鱼和少量种植。在远处,您会看到冰川的闪光。走近一点,你会发现它正在快速融化,几乎要筑成一个巨大的湖泊。当您观看时,湖水破土而出,顺着河床冲下,将河床冲刷得更深,溅起水花溅到远处景观的悬崖上——其中一些已经坍塌。

Imagine yourself skimming over a landscape thousands of years ago, looking at the people below. They’re barely scraping out a living in a hostile territory, doing some hunting, some fishing, and a little planting. Off in the distance, you see the glitter of a glacier. Moving closer, you see that it’s melting fast and that it’s barely damming a huge lake. As you watch, the lake breaks through, sweeping down a riverbed, carving it deeper, splashing up against cliffs on the far side of the landscape—some of which collapse.

当您观看时,茫然的居民开始探索开口。在另一边,有一片郁郁葱葱的风景,与他们从未见过的体型更大的动物成群结队,有些在长着巨大种子头的草地上吃草,有些则在一堆堆掉落的水果上争吵。

As you watch, the dazed inhabitants begin to explore the opening. On the other side, there’s a lush landscape, teaming with bigger animals than they’ve ever seen before, some grazing on grass with huge seed heads, some squabbling over mounds of fallen fruit.

人们搬进来了。几乎立刻,他们开始过上更好的生活。但是随着岁月的流逝,您会看到他们适应了。他们开始用网在湍急的溪流中捕鱼。他们学会了打倒大型动物所需的团队合作,尽管一路上也有一些人死亡。他们找到了更好的方法来种植这种他们称之为“小麦”的新草。

People move in. Almost immediately, they begin to live better. But as the years fly past, you see them adapt. They begin to use nets to fish in the fast-running streams. They learn the teamwork needed to bring down the larger animals, though not without a few deaths along the way. They find ever-better ways to cultivate this new grass they’ve come to call “wheat.”

正如你所看到的,疯狂的创新让位于稳定的解决方案,一种在这片新土地上生活的好方法,一种传授给每一代新人的方法。虽然就在那边,你发现有人在发明轮子......

As you watch, the mad burst of innovation gives way to a stable solution, a good way to live in this new land, a way that’s taught to each new generation. Although just over there, you spy someone inventing the wheel ...

在本世纪初,敏捷方法的采用有时看起来像是一个巨大的堤坝决堤,开辟了一条通向更好——更高效、更快乐——的软件开发方式的途径。许多早期采用者立即看到了好处,即使他们几乎不知道自己在做什么。

In the early years of this century, the adoption of Agile methods sometimes seemed like a vast dam breaking, opening up a way to a better—more productive, more joyful—way of developing software. Many early adopters saw benefits right away, even though they barely knew what they were doing.

有些人比其他人更容易。程序员就像上面寓言中的猎人。是的,他们必须学习新技能才能猎杀野牛,但他们或多或少知道如何猎杀兔子,而且附近有很多兔子。测试人员更像是在用鱼叉捕鱼不起作用的土地上的鱼叉捕鱼者。从用鱼叉捕鱼到用网捕鱼是一个比从兔子到野牛更大的概念跳跃。而且,虽然某些技能(例如清理鱼)在新土地上是相同的,但测试人员必须发明新的织网技能才能真正发挥作用。

Some had an easier time of it than others. Programmers were like the hunters in the fable above. Yes, they had to learn new skills in order to hunt bison, but they knew how to hunt rabbits, more or less, and there were plenty of rabbits around. Testers were more like spear-fishers in a land where spear-fishing wouldn’t work. Going from spear-fishing to net-fishing is a much bigger conceptual jump than going from rabbit to bison. And, while some of the skills—cleaning fish, for example—were the same in the new land, the testers had to invent new skills of net-weaving before they could truly pull their weight.

所以测试滞后了。幸运的是,我们有像 Lisa 和 Janet 这样的早期采用者,他们与程序员、不嫉妒自己的角色或独立性的测试人员一起投入其中,他们是非常令人愉快的人,他们能够找出敏捷测试中最大的变化:测试人员的新的社会角色。

So testing lagged behind. Fortunately, we had early adopters like Lisa and Janet, people who dove right in alongside the programmers, testers who were not jealous of their role or their independence, downright pleasant people who could figure out the biggest change of all in Agile testing: the tester’s new social role.

结果,我们有了这本书。这是稳定的解决方案,是测试人员在我们这个新的敏捷领域生活的好方法。这不是最后的决定——我们可以使用轮子,我自己也渴望有人发明抗生素——但这里所教的内容将对你有用,直到有人,也许是丽莎和珍妮特,带来下一个巨大的变化。

As a result, we have this book. It’s the stable solution, the good way for testers to live in this new Agile land of ours. It’s not the final word—we could use the wheel, and I myself am eager for someone to invent antibiotics—but what’s taught here will serve you well until someone, perhaps Lisa and Janet, brings the next big change.

前言

Preface

我们是极限编程 (XP) 的早期采用者,在 XP 团队中进行测试,这些团队根本不确定测试人员或他们的测试品牌适合在哪里。当时,敏捷(不称为agile yet)有关验收测试或专业测试人员如何做出贡献的文献。我们不仅从自己的经验中学习,而且从小型敏捷社区中的其他人那里学习。2002 年,在Janet 的大力帮助下,Lisa 与 Tip House 合着了《测试极限编程》 。从那时起,敏捷开发得到发展,敏捷测试社区蓬勃发展。有这么多人贡献想法,我们学到了很多关于敏捷测试的知识。

We were early adopters of Extreme Programming (XP), testing on XP teams that weren’t at all sure where testers or their brand of testing fit in. At the time, there wasn’t much in the agile (which wasn’t called agile yet) literature about acceptance testing, or how professional testers might contribute. We learned not only from our own experiences but from others in the small agile community. In 2002, Lisa co-wrote Testing Extreme Programming with Tip House, with lots of help from Janet. Since then, agile development has evolved, and the agile testing community has flourished. With so many people contributing ideas, we’ve learned a whole lot more about agile testing.

我们单独和一起帮助团队过渡到敏捷,帮助测试人员学习如何为敏捷团队做出贡献,并与敏捷社区中的其他人一起探索敏捷团队在测试中更成功的方法。我们的经历不同。Lisa 大部分时间都在稳定的团队中担任敏捷测试员,多年来一直在零售、电话和金融行业的 Web 应用程序上工作。Janet 曾与多个行业的软件组织合作开发企业系统。这些敏捷项目包括开发消息处理系统、环境跟踪系统、远程数据管理系统(包括嵌入式应用程序、通信网络和应用程序)、油气生产核算应用程序、以及在航空运输业中的应用。她扮演过不同的角色——有时是测试员,有时是教练——但她一直致力于更好地将测试员与团队的其他成员结合起来。她在团队中工作的时间短则六个月,长则一年半。

Individually and together, we’ve helped teams transition to agile, helped testers learn how to contribute on agile teams, and worked with others in the agile community to explore ways that agile teams can be more successful at testing. Our experiences differ. Lisa has spent most of her time as an agile tester on stable teams working for years at a time on web applications in the retail, telephony, and financial industries. Janet has worked with software organizations developing enterprise systems in a variety of industries. These agile projects have included developing a message-handling system, an environmental-tracking system, a remote data management system (including an embedded application, with a communication network as well as the application), an oil and gas production accounting application, and applications in the airline transportation industry. She has played different roles—sometimes tester, sometimes coach—but has always worked to better integrate the testers with the rest of the team. She has been with teams from as little as six months to as long as one-and-a-half years.

有了这些不同的观点,我们学会了一起工作并互补彼此的技能,并且我们一起提供了许多演示和教程。

With these different points of view, we have learned to work together and complement each other’s skill sets, and we have given many presentations and tutorials together.

我们为什么写这本书

Why We Wrote This Book

已经出版了几本关于测试和测试模式的敏捷开发的优秀书籍(请参阅我们的参考书目)。这些书通常侧重于帮助开发人员。我们决定写一本书,旨在帮助敏捷团队使用业务可以理解的测试更成功地交付业务价值。我们希望帮助使用更传统的开发方法的测试人员和质量保证 (QA) 专业人员过渡到敏捷开发。

Several excellent books oriented toward agile development on testing and test patterns have been published (see our Bibliography). These books are generally focused on helping the developer. We decided to write a book aimed at helping agile teams be more successful at delivering business value using tests that the business can understand. We want to help testers and quality assurance (QA) professionals who have worked in more traditional development methodologies make the transition to agile development.

我们已经弄清楚如何在实际的、日常的层面上应用我们自己与各种规模的团队合作的经验成果以及其他敏捷实践者的各种想法。我们将所有这些内容汇总在本书中,以帮助测试人员、质量保证经理、开发人员、开发经理、产品所有者以及任何与敏捷项目的有效测试相关的人交付他们的客户需要的软件。然而,我们关注的是测试人员的角色,一个可以被各种专业人士采用的角色。

We’ve figured out how to apply—on a practical, day-to-day level—the fruits of our own experience working with teams of all sizes and a variety of ideas from other agile practitioners. We’ve put all this together in this book to help testers, quality assurance managers, developers, development managers, product owners, and anyone else with a stake in effective testing on agile projects to deliver the software their customers need. However, we’ve focused on the role of the tester, a role that may be adopted by a variety of professionals.

敏捷测试实践并不局限于敏捷团队的成员。它们也可用于改进使用传统开发方法的项目的测试。本书还旨在帮助测试人员使用任何类型的开发方法从事项目。

Agile testing practices aren’t limited to members of agile teams. They can be used to improve testing on projects using traditional development methodologies as well. This book is also intended to help testers working on projects using any type of development methodology.

敏捷开发并不是成功交付软件的唯一途径。然而,我们所处的所有成功团队,无论是敏捷团队还是瀑布团队,都有几个关键的共性。程序员编写并自动化单元和集成测试,以提供良好的代码覆盖率。他们在使用源代码控制和代码集成方面受到纪律处分。熟练的测试人员从开发周期的开始就参与进来,并有时间和资源来完成所有必要形式的测试。定期运行和检查涵盖更高级别系统功能的自动回归套件。开发团队了解客户的工作和需求,并与业务专家密切合作。

Agile development isn’t the only way to successfully deliver software. However, all of the successful teams we’ve been on, agile or waterfall, have had several critical commonalities. The programmers write and automate unit and integration tests that provide good code coverage. They are disciplined in the use of source code control and code integration. Skilled testers are involved from the start of the development cycle and are given time and resources to do an adequate job of all necessary forms of testing. An automated regression suite that covers the system functionality at a higher level is run and checked regularly. The development team understands the customers’ jobs and their needs, and works closely together with the business experts.

使项目成功的是人,而不是方法论或工具。我们喜欢敏捷开发,因为它的价值观、原则和核心实践使人们能够做到最好,而测试和质量是敏捷开发的核心。在本书中,我们将解释如何将敏捷价值观和原则应用于您独特的测试情况,并使您的团队取得成功。我们在第 1 章什么是敏捷测试?”中有更多相关内容。”和第 2 章敏捷测试人员的十项原则”。

People, not methodologies or tools, make projects successful. We enjoy agile development because its values, principles, and core practices enable people to do their best work, and testing and quality are central to agile development. In this book, we explain how to apply agile values and principles to your unique testing situation and enable your teams to succeed. We have more about that in Chapter 1, “What Is Agile Testing, Anyway?” and in Chapter 2, “Ten Principles for Agile Testers.”

我们如何写这本书

How We Wrote This Book

在体验了敏捷开发的好处之后,我们使用敏捷实践来制作这本书。当我们开始写这本书时,我们与来自世界各地的敏捷测试人员和团队进行了交谈,以了解他们遇到了什么问题以及他们是如何解决这些问题的。我们计划了如何在本书中涵盖这些领域。

Having experienced the benefits of agile development, we used agile practices to produce this book. As we began work on the book, we talked to agile testers and teams from around the globe to find out what problems they encountered and how they addressed them. We planned how we would cover these areas in the book.

我们制定了基于两周迭代的发布计划。每两周,我们就会向我们的图书网站发送两章草稿。因为我们不在同一地点,所以我们找到了用于交流的工具,为我们的章节提供“源代码控制”,将产品交付给我们的客户,并获得他们的反馈。我们不能实时“配对”太多,但我们来回交换章节以进行复习和修订,并且每天通过即时消息进行非正式的“站会”。

We made a release plan based on two-week iterations. Every two weeks, we delivered two rough-draft chapters to our book website. Because we aren’t co-located, we found tools to use to communicate, provide “source code control” for our chapters, deliver the product to our customers, and get their feedback. We couldn’t “pair” much real-time, but we traded chapters back and forth for review and revision, and had informal “stand-ups” daily via instant message.

我们的“客户”是敏捷社区中的慷慨人士,他们自愿审阅章节草稿。他们通过电子邮件或(如果我们幸运的话)亲自提供反馈。在我们继续写作和修改时,我们使用反馈来指导我们。在完成所有草稿后,我们制定了一个新的计划来完成修订,并结合了我们“客户”的所有有用想法。

Our “customers” were the generous people in the agile community who volunteered to review draft chapters. They provided feedback by email or (if we were lucky) in person. We used the feedback to guide us as we continued writing and revising. After all the rough drafts were done, we made a new plan to complete the revisions, incorporating all the helpful ideas from our “customers.”

我们最重要的工具是思维导图。我们首先创建了一张思维导图,展示了我们如何设想整本书。然后我们为本书的每个部分创建了思维导图。在写每一章之前,我们用思维导图进行头脑风暴。在我们修订的过程中,我们重新审视了思维导图,这有助于我们思考我们可能遗漏的想法。

Our most important tool was mind maps. We started out by creating a mind map of how we envisioned the whole book. We then created mind maps for each section of the book. Before writing each chapter, we brainstormed with a mind map. As we revised, we revisited the mind maps, which helped us think of ideas we may have missed.

因为我们认为思维导图增加了很多价值,所以我们将思维导图作为每章开头的一部分。我们希望它们能帮助您大致了解本章中包含的所有信息,并启发您自己尝试使用思维导图。

Because we think the mind maps added so much value, we’ve included the mind map as part of the opening of each chapter. We hope they’ll help you get an overview of all the information included in the chapter, and inspire you to try using mind maps yourself.

我们的观众

Our Audience

如果您曾经问​​过以下任何我们已经多次听到的极好的问题,本书将对您有所帮助:

This book will help you if you’ve ever asked any of the following excellent questions, which we’ve heard many times:

图像如果开发人员正在编写测试,那么测试人员会做什么?

If developers are writing tests, what do the testers do?

图像我是一名 QA 经理,我们公司正在实施敏捷开发(Scrum、XP、DSDM,随你喜欢)。我现在的角色是什么?

I’m a QA manager, and our company is implementing agile development (Scrum, XP, DSDM, name your flavor). What’s my role now?

图像 我曾在一个传统的瀑布团队中担任测试员,我对所读到的关于敏捷的内容感到非常兴奋。在敏捷团队中工作我需要知道什么?

I’ve worked as a tester on a traditional waterfall team, and I’m really excited by what I’ve read about agile. What do I need to know to work on an agile team?

图像什么是“敏捷测试员”?

What’s an “agile tester”?

图像我是一个敏捷团队的开发人员。我们正在编写测试优先的代码,但我们的客户仍然对我们提供的产品不满意。我们缺少什么?

I’m a developer on an agile team. We’re writing code test-first, but our customers still aren’t happy with what we deliver. What are we missing?

图像我是一个敏捷团队的开发人员。我们正在编写测试优先的代码。我们确保对所有代码进行测试。为什么我们需要测试人员?

I’m a developer on an agile team. We’re writing our code test-first. We make sure we have tests for all our code. Why do we need testers?

图像我指导一个敏捷开发团队。我们的 QA 团队跟不上我们,测试总是落后。我们应该只计划测试开发背后的迭代吗?

I coach an agile development team. Our QA team can’t keep up with us, and testing always lags behind. Should we just plan to test an iteration behind development?

图像我是一名软件开发经理。我们最近过渡到敏捷,但我们所有的测试人员都退出了。为什么?

I’m a software development manager. We recently transitioned to agile, but all our testers quit. Why?

图像我是一个敏捷团队的测试员。我没有任何编程或自动化技能。在敏捷团队中有适合我的位置吗?

I’m a tester on a team that’s going agile. I don’t have any programming or automation skills. Is there any place for me on an agile team?

图像测试如何跟上两周的迭代?

How can testing possibly keep up with two-week iterations?

图像负载测试、性能测试、可用性测试以及所有其他“能力”呢?这些适合放在哪里?

What about load testing, performance testing, usability testing, all the other “ilities”? Where do these fit in?

图像我们有审计要求。敏捷开发和测试如何解决这些问题?

We have audit requirements. How does agile development and testing address these?

如果您有类似的问题,并且正在寻找有关测试人员如何为敏捷团队做出贡献以及敏捷团队如何有效进行测试的实用建议,那么您选对了这本书。

If you have similar questions and you’re looking for practical advice about how testers contribute to agile teams and how agile teams can do an effective job of testing, you’ve picked up the right book.

敏捷开发有很多“风格”,但它们都有很多共同点。我们支持敏捷宣言,我们在第 1 章什么是敏捷测试?”中对此进行了解释。” 无论您是在练习 Scrum、极限编程、Crystal、DSDM,还是您自己的敏捷开发变体,您都可以在这里找到有助于测试工作的信息。

There are many “flavors” of agile development, but they all have much in common. We support the Agile Manifesto, which we explain in Chapter 1, “What Is Agile Testing, Anyway?” Whether you’re practicing Scrum, Extreme Programming, Crystal, DSDM, or your own variation of agile development, you’ll find information here to help with your testing efforts.

如何使用本书

How to Use This Book

如果您不确定从何处开始阅读本书,或者您只想快速了解一下,我们建议您阅读最后一章,即第 21 章“成功的关键因素”,并按照它引导您的方向进行操作

If you aren’t sure where to start in this book, or you just want a quick overview, we suggest you read the last chapter, Chapter 21, “Key Success Factors,” and follow wherever it leads you.

如果您想快速回答诸如“敏捷测试与瀑布项目测试有何不同?”等问题 或者“传统团队的测试人员和敏捷测试人员有什么区别?”,从第 I 部分开始,其中包括以下章节:

If you want quick answers to questions such as “Is agile testing different than testing on waterfall projects?” or “What’s the difference between a tester on a traditional team and an agile tester?,” start with Part I, which includes the following chapters:

这些章节是 Robin 在他的用户故事中要求的“冰山一角”。它们概述了敏捷与传统的分阶段方法有何不同,并探索了“整个团队”的质量和测试方法。

These chapters are the “tip of the iceberg” that Robin requested in his user story. They include an overview of how agile differs from a traditional phased approach and explore the “whole team” approach to quality and testing.

在本书的这一部分,我们定义了“敏捷测试思维模式”以及是什么让测试人员在敏捷团队中取得成功。我们解释了测试人员如何应用敏捷价值观和原则来贡献他们的特殊专业知识。

In this part of the book we define the “agile testing mind-set” and what makes testers successful on agile teams. We explain how testers apply agile values and principles to contribute their particular expertise.

如果您是传统 QA 团队的测试员或经理,或者您正在指导一个正在转向敏捷的团队,那么第II 部分将帮助您应对转型中的团队所面临的组织挑战。“整个团队”的态度代表了团队成员的许多文化变化,但它有助于克服测试人员在怀疑他们将拥有多少控制权或是否需要编写代码时产生的恐惧。

If you’re a tester or manager on a traditional QA team, or you’re coaching a team that’s moving to agile, Part II will help you with the organizational challenges faced by teams in transition. The “whole team” attitude represents a lot of cultural changes to team members, but it helps overcome the fear testers have when they wonder how much control they’ll have or whether they’ll be expected to write code.

第二部分回答的一些问题是:

Some of the questions answered in Part II are:

图像我们如何让质量检查团队参与进来?

How can we engage the QA team?

图像管理层的期望如何?

What about management’s expectations?

图像我们应该如何构建我们的敏捷团队,测试人员适合在哪里?

How should we structure our agile team, and where do the testers fit?

图像招聘敏捷测试人员时我们看重什么?

What do we look for when hiring an agile tester?

图像我们如何应对分布在全球各地的团队?

How do we cope with a team distributed across the globe?

第二部分还介绍了一些我们并不总是喜欢谈论的话题。我们探索有关如何转换传统环境中常见的流程和模型的想法,例如审计或 SOX 合规性。

Part II also introduces some topics we don’t always enjoy talking about. We explore ideas about how to transition processes and models, such as audits or SOX compliance, that are common in traditional environments.

指标及其应用方式可能是一个有争议的问题,但有一些积极的方法可以使用它们来使团队受益。缺陷跟踪很容易成为团队争论的焦点,诸如“我们是否使用缺陷跟踪系统?”之类的问题。或者“我们什么时候记录错误?”

Metrics and how they’re applied can be a controversial issue, but there are positive ways to use them to benefit the team. Defect tracking easily becomes a point of contention for teams, with questions such as “Do we use a defect-tracking system?” or “When do we log bugs?”

具有传统测试团队经验的人员关于敏捷测试的两个常见问题是“测试计划如何?” 和“真的没有关于敏捷项目的文档吗?” 第二部分揭开这些谜团。

Two common questions about agile testing from people with traditional test team experience are “What about test plans?” and “Is it true there’s no documentation on agile projects?” Part II clears up these mysteries.

第二部分各章如下:

The chapters in Part II are as follows:

您是否需要有关在敏捷项目上进行哪些类型的测试的更多详细信息?你想知道谁做了什么测试吗?你怎么知道你是否已经完成了所有需要的测试?您如何决定哪些实践、技术和工具适合您的特定情况?如果这些是您的顾虑,请查看第 III 部分

Do you want more details on what types of testing are done on agile projects? Are you wondering who does what testing? How do you know whether you’ve done all the testing that’s needed? How do you decide what practices, techniques, and tools fit your particular situation? If these are your concerns, check out Part III.

我们使用 Brian Marick 的敏捷测试象限来解释测试的目的。象限可帮助您定义测试应解决的所有不同领域,从单元级测试到可靠性和其他“能力”,以及介于两者之间的一切。这就是我们深入了解如何交付高质量产品的关键所在。我们解释了可以帮助您与客户进行良好沟通并更好地了解他们的需求的技术。本书的这一部分展示了测试如何在多个层面推动开发。它还为您的工具包提供了工具,可以帮助您有效地定义、设计和执行支持团队和评价产品的测试。这些章节包括以下内容:

We use Brian Marick’s Agile Testing Quadrants to explain the purpose of testing. The quadrants help you define all the different areas your testing should address, from unit level tests to reliability and other “ilities,” and everything in between. This is where we get down into the nitty-gritty of how to deliver a high-quality product. We explain techniques that can help you to communicate well with your customers and better understand their requirements. This part of the book shows how tests drive development at multiple levels. It also provides tools for your toolkit that can help you to effectively define, design, and execute tests that support the team and critique the product. The chapters include the following:

测试自动化是成功的敏捷团队的核心焦点,它对很多人来说都是一个可怕的话题(我们知道,因为它让我们之前感到害怕!)。您如何将测试自动化压缩到短迭代中并仍然完成所有故事?

Test automation is a central focus of successful agile teams, and it’s a scary topic for lots of people (we know, because it’s had us running scared before!). How do you squeeze test automation into short iterations and still get all the stories completed?

第 IV 部分详细介绍了何时以及为何进行自动化、如何克服测试自动化的障碍以及如何开发和实施适合您的团队的测试自动化策略。由于测试自动化工具变化和发展如此迅速,我们的目的不是解释如何使用特定工具,而是帮助您选择和使用适合您情况的正确工具。我们的敏捷测试自动化技巧将帮助您应对困难的挑战,例如测试遗留代码。

Part IV gets into the details of when and why to automate, how to overcome barriers to test automation, and how to develop and implement a test automation strategy that works for your team. Because test automation tools change and evolve so rapidly, our aim is not to explain how to use specific tools, but to help you select and use the right tools for your situation. Our agile test automation tips will help you with difficult challenges such as testing legacy code.

章节如下:

The chapters are as follows:

如果您只是想了解测试人员在整个敏捷开发周期中所做的事情,或者您需要帮助整理本书中的所有信息,请转到第 V 部分。在这里,我们记录了敏捷测试人员生活中的一次迭代,以及更多。测试人员在整个敏捷软件开发周期中贡献了巨大的价值。在第五部分,我们解释了测试人员每天进行的活动。我们从规划发布和迭代开始,让每个迭代都有一个良好的开端,然后通过迭代——与客户和开发团队协作、测试和编写代码。我们通过交付新功能并为团队寻找改进流程的方法来结束迭代。

If you just want to get a feel for what testers do throughout the agile development cycle, or you need help putting together all the information in this book, go to Part V. Here we chronicle an iteration, and more, in the life of an agile tester. Testers contribute enormous value throughout the agile software development cycles. In Part V, we explain the activities that testers do on a daily basis. We start with planning releases and iterations to get each iteration off to a good start, and move through the iteration—collaborating with the customer and development teams, testing, and writing code. We end the iteration by delivering new features and finding ways for the team to improve the process.

这些章节是这样分解的:

The chapters break down this way:

第 21 章关键成功因素”中,我们介绍了敏捷团队可用于成功测试的七个关键因素。如果您在决定从哪里开始敏捷测试或如何改进您现在所做的事情时遇到困难,这些成功因素将为您指明方向。

In Chapter 21, “Key Success Factors,” we present seven key factors agile teams can use for successful testing. If you’re having trouble deciding where to start with agile testing, or how to work on improving what you’re doing now, these success factors will give you some direction.

其他元素

Other Elements

我们还提供了一个词汇表,希望您能从中找到有用的词汇表,以及参考书目中的书籍、文章、网站和博客。

We’ve also included a glossary we hope you will find useful, as well as references to books, articles, websites, and blogs in the bibliography.

就从今天开始吧!

Just Start Doing It—Today!

敏捷开发就是把工作做到最好。每个团队都有独特的挑战。我们已尝试提供所有我们认为可能对敏捷测试人员、他们的团队、经理和客户有帮助的信息。应用您认为适合您情况的技巧。不断试验,评估结果,然后回过头来看这本书,看看有什么可以帮助你改进的。我们的目标是帮助测试人员和敏捷团队尽其所能交付最好、最有价值的产品。

Agile development is all about doing your best work. Every team has unique challenges. We’ve tried to present all the information that we think may help agile testers, their teams, managers, and customers. Apply the techniques that you think are appropriate for your situation. Experiment constantly, evaluate the results, and come back to this book to see what might help you improve. Our goal is to help testers and agile teams enjoy delivering the best and most valuable product they can.

当我们询问 Canoo WebTest 的创始人兼项目经理 Dierk König 时,他认为敏捷测试的首要成功因素是什么,他回答说:“从今天开始!” 您现在可以迈出一小步来改进团队的测试。开始吧!

When we asked Dierk König, founder and project manager of Canoo WebTest, what he thought was the number one success factor for agile testing, he answered: “Start doing it—today!” You can take a baby step to improve your team’s testing right now. Go get started!

致谢

Acknowledgments

有太多人帮助我们完成了这本书,以至于很难知道首先要感谢谁。Chris Guzikowski 给了我们写这本书的机会,并一直鼓励我们。当我们决定是否承担如此艰巨的任务时,迈克·科恩 (Mike Cohn) 给了我们一个明智的建议,即写书的最佳理由是您有话要说。关于敏捷测试,我们肯定有很多话要说。幸运的是,还有很多其他愿意帮助我们的人。

So many people have helped us with this book that it’s hard to know whom to thank first. Chris Guzikowski gave us the opportunity to write this book and kept encouraging us along the way. When we were deciding whether to take on such a mammoth task, Mike Cohn gave us the sage advice that the best reason to write a book is that you have something to say. We sure have lots to say about agile testing. Fortunately, so do lots of other people who were willing to lend us a hand.

非常感谢 Brian Marick 和 Mike Cohn 撰写了如此友善的前言。我们很荣幸 Mike 选择我们的书作为他的签名系列。我们感谢他在本书中包含的许多想法和观察。

Many thanks to Brian Marick and Mike Cohn for writing such kind forewords. We’re honored that Mike selected our book for his signature series. We’re grateful for the many ideas and observations of his that are included in this book.

Brian Marick 的“敏捷测试矩阵”多年来一直在指导我们的敏捷项目,它提供了第 III 部分的核心内容。感谢您,Brian,感谢您提出象限(以及对敏捷测试的许多其他贡献)并让我们在这里使用它们。

Brian Marick’s “Agile Testing Matrix” has guided both of us in our agile projects for several years, and it provides the core of Part III. Thank you, Brian, for thinking up the quadrants (and so many other contributions to agile testing) and letting us use them here.

我们不断利用反馈的敏捷价值。非常感谢我们的官方审稿人:Jennitta Andrea、Gerard Meszaros、Ron Jeffries 和 Paul Duvall。每个人都有独特而有见地的评论,帮助我们大大改进了本书。Gerard 还帮助我们在测试术语方面更加一致和正确,并贡献了一些敏捷测试的成功案例。

We made constant use of the agile value of feedback. Many thanks to our official reviewers: Jennitta Andrea, Gerard Meszaros, Ron Jeffries, and Paul Duvall. Each one had unique and insightful comments that helped us greatly improve the book. Gerard also helped us be more consistent and correct in our testing terminology, and contributed some agile testing success stories.

特别感谢两位审阅者和一流的敏捷测试人员,他们阅读了我们写的每一个字,并花费数小时亲自与我们讨论了章节草稿:Pierre Veragen 和 Paul Rogers。本书中的许多好点子都是他们的。

Special thanks to two reviewers and top-notch agile testers who read every word we wrote and spent hours discussing the draft chapters with us in person: Pierre Veragen and Paul Rogers. Many of the good ideas in this book are theirs.

我们采访了几个团队,了解他们会给新的敏捷团队和测试人员什么建议,并从敏捷测试社区的同事那里征求成功案例和“经验教训”。衷心感谢我们的侧边栏和引用的众多贡献者,以及有用反馈的提供者,包括(排名不分先后)Robin Dymond、Bret Pettichord、Tae Chang、Bob Galen、Erika Boyer、Grig Gheorghiu、Erik Bos、Mark Benander,乔纳森·拉斯穆森、安迪·波尔斯、迪尔克·科尼格、拉斐尔·桑托斯、杰森·霍尔泽、克里斯托夫·卢维翁、大卫·里德、约翰·沃里斯、克里斯·麦克马洪、德克兰·惠兰、迈克尔·博尔顿、伊丽莎白·亨德里克森、乔·亚基奇、安德鲁·格洛弗、亚历山德罗·科里诺、科尼·塔塔利亚、马库斯·加特纳, 梅根·萨姆雷尔, 内森·西尔伯曼, 迈克·托马斯, 迈克·布斯, 史蒂夫·珀金斯, 约瑟夫·金, 雅库布·奥莱兹凯维奇, 皮埃尔·韦拉根 (再次), 保罗·罗杰斯 (再次),

We interviewed several teams to learn what advice they would give new agile teams and testers, and solicited success stories and “lessons learned” from colleagues in the agile testing community. Heartfelt thanks to our many contributors of sidebars and quotes, as well as providers of helpful feedback, including (in no particular order) Robin Dymond, Bret Pettichord, Tae Chang, Bob Galen, Erika Boyer, Grig Gheorghiu, Erik Bos, Mark Benander, Jonathan Rasmusson, Andy Pols, Dierk König, Rafael Santos, Jason Holzer, Christophe Louvion, David Reed, John Voris, Chris McMahon, Declan Whelan, Michael Bolton, Elisabeth Hendrickson, Joe Yakich, Andrew Glover, Alessandro Collino, Coni Tartaglia, Markus Gärtner, Megan Sumrell, Nathan Silberman, Mike Thomas, Mike Busse, Steve Perkins, Joseph King, Jakub Oleszkiewicz, Pierre Veragen (again), Paul Rogers (again), Jon Hagar, Antony Marcano, Patrick Wilson-Welsh, Patrick Fleisch, Apurva Chandra, Ken De Souza, and Carol Vaage.

也非常感谢我们社区的其他非官方审稿人,他们阅读了章节,提供了反馈和想法,并让我们从他们身上汲取灵感,包括 Tom Poppendieck、Jun Bueno、Kevin Lawrence、Hannu Kokko、Titus Brown、Wim van de Goor , Lucas Campos, Kay Johansen, Adrian Howard, Henrik Kniberg, Shelly Park, Robert Small, Senaka Suriyaachchi, and Erik Petersen. 如果我们忘记在这里列出您,那并不是因为我们对您的贡献的重视程度有所降低,只是因为我们没有保留足够好的笔记!我们希望您会在完成的书中看到您的时间和努力是如何得到回报的。

Many thanks also to the rest of our community of unofficial reviewers who read chapters, gave feedback and ideas, and let us bounce ideas off of them, including Tom Poppendieck, Jun Bueno, Kevin Lawrence, Hannu Kokko, Titus Brown, Wim van de Goor, Lucas Campos, Kay Johansen, Adrian Howard, Henrik Kniberg, Shelly Park, Robert Small, Senaka Suriyaachchi, and Erik Petersen. And if we’ve neglected to list you here, it’s not that we value your contribution any less, it’s just that we didn’t keep good enough notes! We hope you will see how your time and effort paid off in the finished book.

我们感谢帮助我们和我们的团队通过敏捷取得成功的敏捷先驱们奠定的基础。您会在参考书目中找到他们的一些作品。我们感谢敏捷团队,他们为我们提供了如此多的开源测试工具,帮助我们所有的团队交付如此多的价值。其中一些工具也列在参考书目中。

We appreciate the groundwork laid by the agile pioneers who have helped us and our teams succeed with agile. You’ll find some of their works in the bibliography. We are grateful for the agile teams that have given us so many open source test tools that help all of our teams deliver so much value. Some of those tools are also listed in the bibliography.

感谢 Mike Thomas 拍摄了本书中出现的许多敏捷团队的动作照片。我们希望这些照片向那些刚接触敏捷测试和开发的人表明,没有什么大奥秘——只是好人聚在一起讨论、演示和画图。

Thanks to Mike Thomas for taking many of the action photos of an agile team that appear in this book. We hope these photos show those of you new to agile testing and development that there’s no big mystery—it’s just good people getting together to discuss, demo, and draw pictures.

非常感谢我们的 Addison-Wesley 编辑和制作团队,他们耐心地回答了许多问题,并将这本书变成了您在这里看到的具有专业外观的书,其中包括 Raina Chrobak、Chris Zahn、John Fuller、Sally Gregg、Bonnie Granat、Diane Freed、Jack刘易斯和金阿尼。

Thanks so much to our Addison-Wesley editorial and production team who patiently answered many questions and turned this into the professional-looking book you see here, including Raina Chrobak, Chris Zahn, John Fuller, Sally Gregg, Bonnie Granat, Diane Freed, Jack Lewis, and Kim Arney.

关于作者

About the Authors

Lisa Crispin是一名敏捷测试实践者和教练。她擅长向测试人员和敏捷团队展示测试人员如何增加价值以及如何通过面向业务的测试来指导开发。她的使命是为软件测试世界带来敏捷的乐趣,为敏捷开发世界带来测试的乐趣。Lisa 于 2000 年加入了她的第一个敏捷团队,多年来担任过程序员、分析师、测试员和 QA 主管。自 2003 年以来,她一直是 ePlan Services, Inc. 的 Scrum/XP 团队的测试员。她经常在北美和欧洲的会议上主持有关敏捷测试的教程和研讨会。Lisa 定期为Better Software杂志、 IEEE SoftwareMethods and Tools等出版物撰写有关敏捷测试的文章. Lisa 还与 Tip House 合着了《测试极限编程》(Addison-Wesley,2002 年)。

Lisa Crispin is an agile testing practitioner and coach. She specializes in showing testers and agile teams how testers can add value and how to guide development with business-facing tests. Her mission is to bring agile joy to the software testing world and testing joy to the agile development world. Lisa joined her first agile team in 2000, having enjoyed many years working as a programmer, analyst, tester, and QA director. Since 2003, she’s been a tester on a Scrum/XP team at ePlan Services, Inc. She frequently leads tutorials and workshops on agile testing at conferences in North America and Europe. Lisa regularly contributes articles about agile testing to publications such as Better Software magazine, IEEE Software, and Methods and Tools. Lisa also co-authored Testing Extreme Programming (Addison-Wesley, 2002) with Tip House.

有关 Lisa 工作的更多信息,请访问她的网站www.lisacrispin.comwww.agiletester.ca,或发送电子邮件至lisa@agiletester.ca

For more about Lisa’s work, visit her websites, www.lisacrispin.com and www.agiletester.ca, or email her at lisa@agiletester.ca.

Janet Gregory是敏捷质量流程咨询和培训公司 DragonFire Inc. 的创始人。她热衷于帮助团队建立质量体系。在过去的十年里,她担任过教练和测试员,将敏捷实践引入了大公司和小公司。她的重点是与业务用户和测试人员合作,以了解他们在敏捷项目中的角色。当 Janet 与她的敏捷团队中的开发人员合作实施创新的敏捷测试自动化解决方案时,她的编程背景无疑是一个优势。Janet 经常在敏捷和测试软件会议上发表演讲,她是北美敏捷测试社区的主要贡献者。

Janet Gregory is the founder of DragonFire Inc., an agile quality process consultancy and training firm. Her passion is helping teams build quality systems. For the past ten years, she has worked as a coach and tester, introducing agile practices into both large and small companies. Her focus is working with business users and testers to understand their role in agile projects. Janet’s programming background is a definite plus when she partners with developers on her agile teams to implement innovative agile test automation solutions. Janet is a frequent speaker at agile and testing software conferences, and she is a major contributor to the North American agile testing community.

有关 Janet 工作的更多信息,请访问她的网站www.janetgregory.cawww.janetgregory.blogspot.comwww.agiletester.ca,或者您可以发送电子邮件至janet@agiletester.ca

For more about Janet’s work, visit her websites at www.janetgregory.ca, www.janetgregory.blogspot.com, and www.agiletester.ca, or you can email her at janet@agiletester.ca.

第一部分 简介

Part I Introduction

在前两章中,我们概述了敏捷测试,强调了敏捷测试与传统的分阶段或“瀑布”方法中的测试有何不同。我们探索“整个团队”的质量和测试方法。

In the first two chapters, we provide an overview of agile testing, highlighting how agile testing differs from testing in a traditional phased or “waterfall” approach. We explore the “whole team” approach to quality and testing.

第 1 章 什么是敏捷测试?

Chapter 1 What Is Agile Testing, Anyway?

图像

就像许多术语一样,“敏捷开发”和“敏捷测试”对不同的人有不同的含义。在本章中,我们解释了我们对敏捷的看法,它反映了敏捷宣言以及不同敏捷方法所共有的一般原则和价值观。我们想与您(读者)分享共同语言,因此我们将复习一些词汇。我们将敏捷开发和测试与更传统的分阶段方法进行比较和对比。敏捷开发提倡的“整个团队”方法是我们对质量和测试的态度的核心,所以我们在这里也谈到了这一点。

Like a lot of terminology, “agile development” and “agile testing” mean different things to different people. In this chapter, we explain our view of agile, which reflects the Agile Manifesto and general principles and values shared by different agile methods. We want to share a common language with you, the reader, so we’ll go over some of our vocabulary. We compare and contrast agile development and testing with the more traditional phased approach. The “whole team” approach promoted by agile development is central to our attitude toward quality and testing, so we also talk about that here.

敏捷价值观

Agile Values

“敏捷”是一个流行词,可能有一天会不再使用,并使这本书看起来过时了。它具有适用于不同情况的不同含义。定义“敏捷开发”的一种方法是查看敏捷宣言(见图1-1)。

“Agile” is a buzzword that will probably fall out of use someday and make this book seem obsolete. It’s loaded with different meanings that apply in different circumstances. One way to define “agile development” is to look at the Agile Manifesto (see Figure 1-1).

图 1-1敏捷宣言

Figure 1-1 Agile Manifesto

图像

使用宣言中的价值观来指导我们,我们努力在极短的发布周期内提供小块的商业价值。

Using the values from the Manifesto to guide us, we strive to deliver small chunks of business value in extremely short release cycles.

我们在本书中从广义上使用“敏捷”一词。无论您的团队是在实践一种特定的敏捷方法,例如 Scrum、XP、Crystal、DSDM 或 FDD,仅举几例,还是只是采用对您的情况有意义的任何原则和实践,您都应该能够将这些想法应用到这本书。如果您使用高质量的软件及时为企业提供价值,并且您的团队不断努力改进,您会在这里找到有用的信息。同时,我们认为某些特定的敏捷实践对任何团队的成功都至关重要。我们将在整本书中讨论这些。

We use the word “agile” in this book in a broad sense. Whether your team is practicing a particular agile method, such as Scrum, XP, Crystal, DSDM, or FDD, to name a few, or just adopting whatever principles and practices make sense for your situation, you should be able to apply the ideas in this book. If you’re delivering value to the business in a timely manner with high-quality software, and your team continually strives to improve, you’ll find useful information here. At the same time, there are particular agile practices we feel are crucial to any team’s success. We’ll talk about these throughout the book.

第 21 章,“关键成功因素”,列出了敏捷测试的关键成功因素。

Chapter 21, “Key Success Factors,” lists key success factors for agile testing.

“敏捷测试”是什么意思?

What Do We Mean by “Agile Testing”?

您可能已经注意到,我们使用术语“测试人员”来描述主要活动围绕测试和质量保证展开的人员。您还会看到,我们经常使用“程序员”一词来描述主要活动围绕编写生产代码的人。我们并不打算让这些术语听起来狭隘或无关紧要。程序员所做的不仅仅是将规范转化为程序。我们不称他们为“开发人员”,因为每个人参与交付软件的是开发人员。测试人员所做的不仅仅是执行“测试任务”。每个敏捷团队成员都专注于交付提供商业价值的高质量产品。敏捷测试人员致力于确保他们的团队交付客户所需的质量。为方便起见,我们使用术语“程序员”和“测试人员”。

You might have noticed that we use the term “tester” to describe a person whose main activities revolve around testing and quality assurance. You’ll also see that we often use the word “programmer” to describe a person whose main activities revolve around writing production code. We don’t intend that these terms sound narrow or insignificant. Programmers do more than turn a specification into a program. We don’t call them “developers,” because everyone involved in delivering software is a developer. Testers do more than perform “testing tasks.” Each agile team member is focused on delivering a high-quality product that provides business value. Agile testers work to ensure that their team delivers the quality their customers need. We use the terms “programmer” and “tester” for convenience.

敏捷团队使用的几个核心实践与测试有关。敏捷程序员使用测试驱动开发 (TDD)(也称为测试驱动设计)来编写高质量的生产代码。使用 TDD,程序员为一小部分功能编写测试,看到它失败,编写使其通过的代码,然后继续下一个小功能。程序员还编写代码集成测试以确保小代码单元按预期协同工作。这一基本实践已被许多团队采用,甚至是那些不称自己为“敏捷”的团队,因为它只是一种思考软件设计和防止缺陷的聪明方法。图 1-2显示了程序员可能会看到的示例单元测试结果。

Several core practices used by agile teams relate to testing. Agile programmers use test-driven development (TDD), also called test-driven design, to write quality production code. With TDD, the programmer writes a test for a tiny bit of functionality, sees it fail, writes the code that makes it pass, and then moves on to the next tiny bit of functionality. Programmers also write code integration tests to make sure the small units of code work together as intended. This essential practice has been adopted by many teams, even those that don’t call themselves “agile,” because it’s just a smart way to think through your software design and prevent defects. Figure 1-2 shows a sample unit test result that a programmer might see.

图 1-2示例单元测试输出

Figure 1-2 Sample unit test output

图像

本书与单元级或组件级测试无关,但这些类型的测试对于成功的项目至关重要。Brian Marick [2003] 将这些类型的测试描述为“支持团队”,帮助程序员知道接下来要编写什么代码。Brian 还创造了术语“面向技术的测试”,即属于程序员领域并使用程序员术语和行话描述的测试。在第二部分中,我们介绍了敏捷测试象限并检查了敏捷测试的不同类别。如果您想了解有关编写单元和组件测试以及 TDD 的更多信息,参考书目将引导您找到一些不错的资源。

This book isn’t about unit-level or component-level testing, but these types of tests are critical to a successful project. Brian Marick [2003] describes these types of tests as “supporting the team,” helping the programmers know what code to write next. Brian also coined the term “technology-facing tests,” tests that fall into the programmer’s domain and are described using programmer terms and jargon. In Part II, we introduce the Agile Testing Quadrants and examine the different categories of agile testing. If you want to learn more about writing unit and component tests, and TDD, the bibliography will steer you to some good resources.

如果您想了解应用于测试的敏捷价值观、原则和实践如何帮助您作为测试人员做到最好,并帮助您的团队交付更多业务价值,请继续阅读。如果你费心拿起这本书,你可能是那种不断努力成长和学习的专业人士。您可能拥有优秀的敏捷团队成功所需的心态。本书将向您展示改进组织产品、为团队提供最大价值并享受工作的方法。

If you want to know how agile values, principles, and practices applied to testing can help you, as a tester, do your best work, and help your team deliver more business value, please keep reading. If you’ve bothered to pick up this book, you’re probably the kind of professional who continually strives to grow and learn. You’re likely to have the mind-set that a good agile team needs to succeed. This book will show you ways to improve your organization’s product, provide the most value possible to your team, and enjoy your job.

丽莎的故事

Lisa’s Story

在完成本章的工作间隙,我与一位在一家大公司从事质量保证工作的朋友交谈。那是一年中的繁忙时期,管理层希望每个人都加班。他说,“如果我认为加班 100 小时可以解决我们的问题,我会每晚工作到 7 点,直到完成。但事实是,解决我们的问题可能需要额外 4,000 个小时,所以加班感觉毫无意义。” 这听起来很熟悉吗?

During a break from working on this chapter, I talked to a friend who works in quality assurance for a large company. It was a busy time of year, and management expected everyone to work extra hours. He said, “If I thought working 100 extra hours would solve our problems, I’d work ‘til 7 every night until that was done. But the truth was, it might take 4,000 extra hours to solve our problems, so working extra feels pointless.” Does this sound familiar?

——丽莎

—Lisa

如果您在软件行业工作了很长时间,您可能有机会感觉自己像 Lisa 的朋友。当您的任务无法完成时,更努力和更长时间的工作无济于事。敏捷开发承认这样一个现实,即我们在一天或一周内只有那么多的高效工作时间,并且我们无法计划改变的不可避免性。

If you’ve worked in the software industry long, you’ve probably had the opportunity to feel like Lisa’s friend. Working harder and longer doesn’t help when your task is impossible to achieve. Agile development acknowledges the reality that we only have so many good productive hours in a day or week, and that we can’t plan away the inevitability of change.

敏捷开发鼓励我们作为一个团队来解决我们的问题。业务人员、程序员、测试人员、分析师——参与软件开发的每个人——共同决定如何最好地改进他们的产品。最重要的是,作为测试人员,我们与一群人一起工作,他们都认为有责任提供尽可能最好的质量,并且都专注于测试。我们喜欢做这项工作,您也会喜欢。

Agile development encourages us to solve our problems as a team. Business people, programmers, testers, analysts—everyone involved in software development—decides together how best to improve their product. Best of all, as testers, we’re working together with a team of people who all feel responsible for delivering the best possible quality, and who are all focused on testing. We love doing this work, and you will too.

当我们在本书中提到“敏捷测试”时,我们通常是在谈论面向业务的测试,即定义业务专家所需特性和功能的测试。我们认为“面向客户”是“面向业务”的同义词。本书中的“测试”还包括批评产品的测试,并专注于发现成品中可能缺少的东西,以便我们改进它。它包括单元和组件级别测试之外的几乎所有内容:功能、系统、负载、性能、安全性、压力、可用性、探索性、端到端和用户接受度。所有这些类型的测试都可能适用于任何给定的项目,无论是敏捷项目还是使用更传统方法的项目。

When we say “agile testing” in this book, we’re usually talking about business-facing tests, tests that define the business experts’ desired features and functionality. We consider “customer-facing” a synonym for “business-facing.” “Testing” in this book also includes tests that critique the product and focus on discovering what might be lacking in the finished product so that we can improve it. It includes just about everything beyond unit and component level testing: functional, system, load, performance, security, stress, usability, exploratory, end-to-end, and user acceptance. All these types of tests might be appropriate to any given project, whether it’s an agile project or one using more traditional methodologies.

敏捷测试不仅仅意味着在敏捷项目上进行测试。一些测试方法,例如探索性测试,本质上是敏捷的,无论它是否完成了一个敏捷项目。测试一个应用程序并计划在你进行时了解它,并让这些信息指导你的测试,这与评估工作软件和响应变化是一致的。后面的章节讨论敏捷测试形式以及“敏捷测试”实践。

Agile testing doesn’t just mean testing on an agile project. Some testing approaches, such as exploratory testing, are inherently agile, whether it’s done an agile project or not. Testing an application with a plan to learn about it as you go, and letting that information guide your testing, is in line with valuing working software and responding to change. Later chapters discuss agile forms of testing as well as “agile testing” practices.

敏捷团队角色和活动的一点背景

A Little Context for Roles and Activities on an Agile Team

我们将在本书中大量讨论“客户团队”和“开发团队”。它们之间的区别在于它们在交付产品时所具备的技能。

We’ll talk a lot in this book about the “customer team” and the “developer team.” The difference between them is the skills they bring to delivering a product.

客户团队

Customer Team

客户团队包括业务专家、产品所有者、领域专家、产品经理、业务分析师、主题专家——项目“业务”方面的每个人。客户团队编写开发团队交付的故事或功能集。他们提供的示例将以面向业务的测试形式推动编码。他们在每次迭代中与开发团队沟通和协作,回答问题,在白板上绘制示例,以及审查完成的故事或故事的一部分。

The customer team includes business experts, product owners, domain experts, product managers, business analysts, subject matter experts—everyone on the “business” side of a project. The customer team writes the stories or feature sets that the developer team delivers. They provide the examples that will drive coding in the form of business-facing tests. They communicate and collaborate with the developer team throughout each iteration, answering questions, drawing examples on the whiteboard, and reviewing finished stories or parts of stories.

测试人员是客户团队不可或缺的成员,帮助引出需求和示例,并帮助客户将他们的需求表达为测试。

Testers are integral members of the customer team, helping elicit requirements and examples and helping the customers express their requirements as tests.

开发团队

Developer Team

参与交付代码的每个人都是开发人员,并且是开发团队的一部分。敏捷原则鼓励团队成员进行多项活动;任何团队成员都可以承担任何类型的任务。许多敏捷实践者不鼓励团队中的专门角色,并鼓励所有团队成员尽可能地将他们的技能传授给其他人。然而,每个团队都需要决定他们的项目需要什么专业知识。程序员、系统管理员、架构师、数据库管理员、技术作家、安全专家以及身兼多职的人员可能是团队的一部分,无论是物理上还是虚拟上。

Everyone involved with delivering code is a developer, and is part of the developer team. Agile principles encourage team members to take on multiple activities; any team member can take on any type of task. Many agile practitioners discourage specialized roles on teams and encourage all team members to transfer their skills to others as much as possible. Nevertheless, each team needs to decide what expertise their projects require. Programmers, system administrators, architects, database administrators, technical writers, security specialists, and people who wear more than one of these hats might be part of the team, physically or virtually.

测试人员也在开发团队中,因为测试是敏捷软件开发的核心组成部分。测试人员代表客户提倡质量,并协助开发团队实现最大的商业价值。

Testers are also on the developer team, because testing is a central component of agile software development. Testers advocate for quality on behalf of the customer and assist the development team in delivering the maximum business value.

客户和开发团队之间的互动

Interaction between Customer and Developer Teams

客户和开发团队始终紧密合作。理想情况下,他们只是一个拥有共同目标的团队。该目标是为组织提供价值。敏捷项目在迭代中取得进展,迭代是通常持续一到四个星期的小开发周期。客户团队在开发人员的输入下,将确定要开发的故事的优先级,而开发团队将确定他们可以承担多少工作。他们将一起工作,通过测试和示例定义需求,并编写使测试通过的代码。测试人员涉足各个领域,了解客户的观点以及技术实施的复杂性(见图1-3)。

The customer and developer teams work closely together at all times. Ideally, they’re just one team with a common goal. That goal is to deliver value to the organization. Agile projects progress in iterations, which are small development cycles that typically last from one to four weeks. The customer team, with input from the developers, will prioritize stories to be developed, and the developer team will determine how much work they can take on. They’ll work together to define requirements with tests and examples, and write the code that makes the tests pass. Testers have a foot in each world, understanding the customer viewpoint as well as the complexities of the technical implementation (see Figure 1-3).

图1-3角色交互

Figure 1-3 Interaction of roles

图像

一些敏捷团队没有任何成员将自己定义为“测试人员”。然而,他们都需要有人帮助客户团队为迭代的故事编写面向业务的测试,确保测试通过,并确保足够的回归测试是自动化的。即使一个团队确实有测试人员,整个敏捷团队也要负责这些测试任务。我们在敏捷团队中的经验表明,测试技能和经验对于项目成功至关重要,并且测试人员确实为敏捷团队增加了价值。

Some agile teams don’t have any members who define themselves as “testers.” However, they all need someone to help the customer team write business-facing tests for the iteration’s stories, make sure the tests pass, and make sure that adequate regression tests are automated. Even if a team does have testers, the entire agile team is responsible for these testing tasks. Our experience with agile teams has shown that testing skills and experience are vital to project success and that testers do add value to agile teams.

敏捷测试有何不同?

How Is Agile Testing Different?

我们都在世纪之交开始在敏捷团队中工作。与许多刚接触敏捷的测试人员一样,我们一开始并不知道会发生什么。我们与各自的敏捷团队一起工作,我们学到了很多关于敏捷项目测试的知识。我们还实施了其他敏捷测试人员和团队建议的想法和实践。多年来,我们也与其他敏捷测试人员分享了我们的经验。我们在敏捷和测试会议上促进了研讨会和指导教程,与本地用户组交谈,并加入了无数关于敏捷测试邮件列表的讨论。通过这些经验,我们确定了敏捷团队测试与传统瀑布式开发项目测试之间的差异。敏捷开发在很多方面改变了测试行业。

We both started working on agile teams at the turn of the millennium. Like a lot of testers who are new to agile, we didn’t know what to expect at first. Together with our respective agile teams, we’ve worked on we’ve learned a lot about testing on agile projects. We’ve also implemented ideas and practices suggested by other agile testers and teams. Over the years, we’ve shared our experiences with other agile testers as well. We’ve facilitated workshops and led tutorials at agile and testing conferences, talked with local user groups, and joined countless discussions on agile testing mailing lists. Through these experiences, we’ve identified differences between testing on agile teams and testing on traditional waterfall development projects. Agile development has transformed the testing profession in many ways.

在传统团队中工作

Working on Traditional Teams

与程序员密切合作或从最早阶段参与项目对我们来说都不是新鲜事。然而,我们习惯于严格执行狭义软件开发生命周期的门控阶段,从发布计划和需求定义开始,通常以匆忙的测试阶段和延迟发布结束。事实上,我们经常被推为看门人的角色,告诉业务经理,“抱歉,要求已冻结;我们可以在下一个版本中添加该功能。”

Neither working closely with programmers nor getting involved with a project from the earliest phases was new to us. However, we were used to strictly enforced gated phases of a narrowly defined software development life cycle, starting with release planning and requirements definition and usually ending with a rushed testing phase and a delayed release. In fact, we often were thrust into a gatekeeper role, telling business managers, “Sorry, the requirements are frozen; we can add that feature in the next release.”

作为质量保证团队的领导者,我们也经常被期望充当质量的守门人。我们无法控制代码的编写方式,甚至无法控制是否有任何程序员测试了他们的代码,除非通过我们个人的协作努力。我们的开发后测试阶段有望在代码完成后提高质量。我们有控制的错觉。我们通常拥有生产的关键,有时我们有权推迟发布或阻止它们继续进行。Lisa 甚至拥有“质量负责人”的称号,但实际上她只是 QA 团队的经理。

As leaders of quality assurance teams, we were also often expected to act as gatekeepers of quality. We couldn’t control how the code was written, or even if any programmers tested their code, other than by our personal efforts at collaboration. Our post-development testing phases were expected to boost quality after code was complete. We had the illusion of control. We usually had the keys to production, and sometimes we had the power to postpone releases or stop them from going forward. Lisa even had the title of “Quality Boss,” when in fact she was merely the manager of the QA team.

我们的开发周期通常很长。生产数据库软件的公司的项目可能会持续一年。Lisa 在一家互联网初创公司经历的六个月发布周期在当时看来很短,尽管冻结需求仍然是一段很长的时间。尽管有很多过程和纪律,在进入下一阶段之前努力完成一个阶段,但竞争有足够的时间领先,而且应用程序并不总是客户所期望的。

Our development cycles were generally long. Projects at a company that produced database software might last for a year. The six-month release cycles Lisa experienced at an Internet start-up seemed short at the time, although it was still a long time to have frozen requirements. In spite of much process and discipline, diligently completing one phase before moving on to the next, it was plenty of time for the competition to come out ahead, and the applications were not always what the customers expected.

传统团队专注于确保所有指定要求都在最终产品中交付。如果在最初的目标发布日期之前一切还没有准备好,发布通常会被推迟。开发团队通常不知道发布中有哪些功能,或者它们应该如何工作。个别程序员倾向于专注于代码的特定区域。测试人员研究需求文档来编写他们的测试计划,然后等待工作交付给他们进行测试。

Traditional teams are focused on making sure all the specified requirements are delivered in the final product. If everything isn’t ready by the original target release date, the release is usually postponed. The development teams don’t usually have input about what features are in the release, or how they should work. Individual programmers tend to specialize in a particular area of the code. Testers study the requirements documents to write their test plans, and then they wait for work to be delivered to them for testing.

在敏捷团队中工作

Working on Agile Teams

过渡到敏捷项目的短迭代可能会产生最初的震惊和敬畏。我们怎么可能定义需求,然后在一、二、三或四个星期内测试和交付生产就绪代码?这对于拥有不同职能的独立团队的大型组织来说尤其困难,对于地理位置分散的团队来说更难。所有这些不同的程序员、测试人员、分析师、项目经理和无数专业人士在新的敏捷项目中处于什么位置?我们怎么可能这么快地编码和测试呢?我们在哪里可以找到时间来进行自动化测试等困难的工作?我们对交付到生产环境的错误代码有什么控制?

Transitioning to the short iterations of an agile project might produce initial shock and awe. How can we possibly define requirements and then test and deliver production-ready code in one, two, three, or four weeks? This is particularly tough for larger organizations with separate teams for different functions and even harder for teams that are geographically dispersed. Where do all these various programmers, testers, analysts, project managers, and countless specialties fit in a new agile project? How can we possibly code and test so quickly? Where would we find time for difficult efforts such as automating tests? What control do we have over bad code getting delivered to production?

我们将分享我们第一次敏捷体验的故事,向您展示每个人都必须从某个地方开始。

We’ll share our stories from our first agile experiences to show you that everyone has to start somewhere.

丽莎的故事

Lisa’s Story

我的第一个敏捷团队采用了极限编程 (XP),并非没有一些“学习经验”。作为一个由八名程序员组成的团队中唯一的专业测试员,他们还没有学会如何自动化单元测试,这令人沮丧。前两周的迭代感觉就像跳下悬崖。

My first agile team embraced Extreme Programming (XP), not without some “learning experiences.” Serving as the only professional tester on a team of eight programmers who hadn’t learned how to automate unit tests was disheartening. The first two-week iteration felt like jumping off a cliff.

幸运的是,我们有一位优秀的教练、出色的培训、一个由敏捷实践者组成的支持社区,他们可以分享想法,也有时间学习。我们一起弄清楚了如何将测试集成到敏捷项目中的一些细节——实际上,如何通过测试驱动项目。我学会了如何利用我的测试技能和经验为敏捷团队增加真正的价值。

Fortunately, we had a good coach, excellent training, a supportive community of agile practitioners with ideas to share, and time to learn. Together we figured out some ins and outs of how to integrate testing into an agile project—indeed, how to drive the project with tests. I learned how I could use my testing skills and experience to add real value to an agile team.

对我(前质量主管)来说,最难了解的事情是客户,而不是我,决定了产品的质量标准。第一次迭代后我惊恐地发现当两个用户同时登录时代码很容易崩溃。我的教练不顾我的强烈反对,耐心地解释说,我们的客户,一家初创公司,希望能够向潜在客户展示功能。可靠性和稳健性还不是问题。

The toughest thing for me (the former Quality Boss) to learn was that the customers, not I, decided on quality criteria for the product. I was horrified after the first iteration to find that the code crashed easily when two users logged in concurrently. My coach patiently explained, over my strident objections, that our customer, a start-up company, wanted to be able to show features to potential customers. Reliability and robustness were not yet the issue.

我了解到我的工作是帮助客户告诉我们在每次迭代中什么对他们有价值,并编写测试以确保这就是他们得到的。

I learned that my job was to help the customers tell us what was valuable to them during each iteration, and to write tests to ensure that’s what they got.

——丽莎

—Lisa

珍妮特的故事

Janet’s Story

我第一次涉足敏捷世界也是一次极限编程 (XP) 活动。我刚来自一个组织,该组织使用一些非常糟糕的实践来实践瀑布,包括给测试团队一天左右的时间来测试六个月的代码。在我担任 QA 经理的下一份工作中,开发经理和我都在学习 XP 的真正含义。我们成功地创建了一个合作良好的团队,并成功地自动化了大部分功能测试。当该组织在互联网泡沫破灭期间裁员时,我发现自己在另一个组织担任新职位,作为一个 XP 项目的大约 10 名开发人员的唯一测试员。

My first foray into the agile world was also an Extreme Programming (XP) engagement. I had just come from an organization that practiced waterfall with some extremely bad practices, including giving the test team a day or so to test six months of code. In my next job as QA manager, the development manager and I were both learning what XP really meant. We successfully created a team that worked well together and managed to automate most of the tests for the functionality. When the organization downsized during the dot-com bust, I found myself in a new position at another organization as the lone tester with about ten developers on an XP project.

在项目的第一天,开发人员之一乔纳森·拉斯穆森 (Jonathan Rasmusson) 来找我,问我为什么会在那里。团队在实践 XP,而程序员在实践测试先行并使他们自己的所有测试自动化。参与其中是我无法抗拒的挑战。团队不知道我能增加什么价值,但我知道我有独特的能力可以帮助团队。那次经历永远改变了我的生活,因为我了解了敏捷项目的细微差别,并确定我一生的工作是让测试人员的角色变得更有成就感。

On my first day of the project, Jonathan Rasmusson, one of the developers, came up to me and asked me why I was there. The team was practicing XP, and the programmers were practicing test-first and automating all their own tests. Participating in that was a challenge I couldn’t resist. The team didn’t know what value I could add, but I knew I had unique abilities that could help the team. That experience changed my life forever, because I gained an understanding of the nuances of an agile project and determined then that my life’s work was to make the tester role a more fulfilling one.

——珍妮特

—Janet

敏捷团队与业务紧密合作,对需求有详细的了解。他们专注于他们可以提供的价值,并且他们可能对确定功能的优先级有大量的投入。测试人员不会坐等工作;他们站起来寻找在整个开发周期及以后做出贡献的方法。

Agile teams work closely with the business and have a detailed understanding of the requirements. They’re focused on the value they can deliver, and they might have a great deal of input into prioritizing features. Testers don’t sit and wait for work; they get up and look for ways to contribute throughout the development cycle and beyond.

如果对敏捷项目的测试感觉就像对传统项目的测试一样,我们就不会觉得有必要写一本书了。让我们比较和对比这些测试方法。

If testing on an agile project felt just like testing on a traditional project, we wouldn’t feel the need to write a book. Let’s compare and contrast these testing methods.

传统与敏捷测试

Traditional vs. Agile Testing

它有助于从查看敏捷测试与传统软件开发中的测试之间的相似性开始。考虑图 1-4

It helps to start by looking at similarities between agile testing and testing in traditional software development. Consider Figure 1-4.

图 1-4传统测试与敏捷测试

Figure 1-4 Traditional testing vs. agile testing

图像

在分阶段方法图中,很明显测试发生在最后,就在发布之前。该图是理想化的,因为它给人的印象是,用于测试的时间与用于编码的时间一样多。在许多项目中,情况并非如此。测试被“压缩”了,因为编码花费的时间比预期的要长,而且团队最终进入了编码和修复的循环。

In the phased approach diagram, it is clear that testing happens at the end, right before release. The diagram is idealistic, because it gives the impression there is as much time for testing as there is for coding. In many projects, this is not the case. The testing gets “squished” because coding takes longer than expected, and because teams get into a code-and-fix cycle at the end.

敏捷是迭代和增量的。这意味着测试人员会在编码完成后立即测试每个编码增量。一次迭代可能短至一周,也可能长至一个月。该团队构建并测试了一些代码,确保其正常工作,然后继续进行下一个需要构建的部分。程序员永远不会领先于测试人员,因为一个故事只有经过测试才算“完成”。我们将在整本书中更多地讨论这一点。

Agile is iterative and incremental. This means that the testers test each increment of coding as soon as it is finished. An iteration might be as short as one week, or as long as a month. The team builds and tests a little bit of code, making sure it works correctly, and then moves on to next piece that needs to be built. Programmers never get ahead of the testers, because a story is not “done” until it has been tested. We’ll talk much more about this throughout the book.

敏捷团队采用的项目方法多种多样。一个团队可能专注于一个项目,也可能是另一个项目的一部分更大的项目。无论您的项目有多大,您仍然需要从某个地方开始。您的团队可能会在估算会议上处理一个史诗或功能,一组相关的故事,或者您可能会开会计划发布。无论项目或项目的子集是如何开始的,您都需要对其有一个高层次的理解。在准备发布时,您可能会想出一个测试计划或策略,但它可能看起来与您之前完成的任何测试计划都大不相同。

There’s tremendous variety in the approaches to projects that agile teams take. One team might be dedicated to a single project or might be part of another bigger project. No matter how big your project is, you still have to start somewhere. Your team might take on an epic or feature, a set of related stories at an estimating meeting, or you might meet to plan the release. Regardless of how a project or subset of a project gets started, you’ll need to get a high-level understanding of it. You might come up with a plan or strategy for testing as you prepare for a release, but it will probably look quite different from any test plan you’ve done before.

每个项目、每个团队,有时甚至每个迭代都是不同的。你的团队如何解决问题应该取决于问题、人员和你可用的工具。作为敏捷团队成员,您需要适应团队的需求。

Every project, every team, and sometimes every iteration is different. How your team solves problems should depend on the problem, the people, and the tools you have available. As an agile team member, you will need to be adaptive to the team’s needs.

与其根据业务分析师在任何人想到编写一行代码之前创建的需求文档创建测试,不如有人需要在编码开始前几天或几小时编写测试来说明每个故事的需求。这通常是一个团队之间的协作努力业务或领域专家以及测试人员、分析师或其他开发团队成员。详细的功能测试用例,最好基于业务专家提供的示例,充实需求。测试人员将进行手动探索性测试,以发现定义的测试用例可能遗漏的重要错误。随着对每个故事的编码进行,测试人员可能会与其他开发人员配对以自动化和执行测试用例。自动功能测试被添加到回归测试套件中。当演示最小功能的测试完成时,团队可以认为故事已经完成。

Rather than creating tests from a requirements document that was created by business analysts before anyone ever thought of writing a line of code, someone will need to write tests that illustrate the requirements for each story days or hours before coding begins. This is often a collaborative effort between a business or domain expert and a tester, analyst, or some other development team member. Detailed functional test cases, ideally based on examples provided by business experts, flesh out the requirements. Testers will conduct manual exploratory testing to find important bugs that defined test cases might miss. Testers might pair with other developers to automate and execute test cases as coding on each story proceeds. Automated functional tests are added to the regression test suite. When tests demonstrating minimum functionality are complete, the team can consider the story finished.

如果你参加过本世纪初的敏捷会议和研讨会,你会听到很多关于 TDD 和验收测试的信息,但很少听到其他关键类型的测试,例如负载、性能、安全性、可用性和其他“能力”测试。作为测试人员,我们认为这有点奇怪,因为所有这些类型的测试对于敏捷项目和使用任何其他开发方法的项目一样重要。真正的区别在于,我们喜欢在开发过程中尽早进行这些测试,以便它们也可以驱动设计和编码。

If you attended agile conferences and seminars in the early part of this decade, you heard a lot about TDD and acceptance testing but not so much about other critical types of testing, such as load, performance, security, usability, and other “ility” testing. As testers, we thought that was a little weird, because all these types of testing are just as vital on agile projects as they are on projects using any other development methodology. The real difference is that we like to do these tests as early in the development process as we can so that they can also drive design and coding.

如果团队像 Lisa 的团队那样实际发布每个迭代,则每个迭代的最后一两天就是“结束游戏”,此时可以进行用户验收测试、培训、错误修复和部署到暂存环境。其他团队,例如 Janet 的团队,每隔几次迭代就发布一次,甚至可能有整个迭代的“结束游戏”活动来验证发布准备情况。这里的区别是所有的测试都没有留到最后。

If the team actually releases each iteration, as Lisa’s team does, the last day or two of each iteration is the “end game,” the time when user acceptance testing, training, bug fixing, and deployments to staging environments can occur. Other teams, such as Janet’s, release every few iterations, and might even have an entire iteration’s worth of “end game” activities to verify release readiness. The difference here is that all the testing is not left until the end.

作为敏捷团队的测试人员,您是将代码发布到生产环境的关键角色,就像您在更传统的环境中一样。您可能会运行脚本或进行手动测试以验证发布的所有元素(例如数据库更新脚本)是否就位。所有团队成员都参与可能针对每次迭代或每次发布进行的回顾或其他流程改进活动。整个团队集思广益解决问题并改进流程和实践的方法。

As a tester on an agile team, you’re a key player in releasing code to production, just as you might have been in a more traditional environment. You might run scripts or do manual testing to verify all elements of a release, such as database update scripts, are in place. All team members participate in retrospectives or other process improvement activities that might occur for every iteration or every release. The whole team brainstorms ways to solve problems and improve processes and practices.

敏捷项目有多种风格。您的团队是否在全新的(新)开发项目中从头开始?如果是这样,与在没有自动回归套件的遗留系统上重写或构建的团队相比,您面临的挑战可能更少。与第三方合作会给任何团队带来额外的测试挑战。

Agile projects have a variety of flavors. Is your team starting with a clean slate, in a greenfield (new) development project? If so, you might have fewer challenges than a team faced with rewriting or building on a legacy system that has no automated regression suite. Working with a third party brings additional testing challenges to any team.

无论您使用哪种开发方式,软件开发生命周期中几乎相同的元素都需要发生。区别敏捷的特点是时间框架大大缩短,活动同时发生。参与者、测试和工具需要具有适应性。

Whatever flavor of development you’re using, pretty much the same elements of a software development life cycle need to happen. The difference with agile is that time frames are greatly shortened, and activities happen concurrently. Participants, tests, and tools need to be adaptive.

敏捷项目中测试人员最关键的区别是来自测试的快速反馈。它推动项目向前发展,如果未达到某些里程碑,没有守门人准备阻止项目进展。

The most critical difference for testers in an agile project is the quick feedback from testing. It drives the project forward, and there are no gatekeepers ready to block project progress if certain milestones aren’t met.

我们遇到过抵制向敏捷开发过渡的测试人员,他们担心“敏捷开发”等同于混乱、缺乏纪律、缺乏文档以及对测试人员不利的环境。虽然一些团队似乎确实使用“敏捷”这个流行语来证明他们想做什么就做什么,但真正的敏捷团队都是关于可重复的质量和效率。根据我们的经验,敏捷团队是测试人员的绝佳去处。

We’ve encountered testers who resist the transition to agile development, fearing that “agile development” equates with chaos, lack of discipline, lack of documentation, and an environment that is hostile to testers. While some teams do seem to use the “agile” buzzword to justify simply doing whatever they want, true agile teams are all about repeatable quality as well as efficiency. In our experience, an agile team is a wonderful place to be a tester.

全团队方法

Whole-Team Approach

敏捷开发与传统开发的最大区别之一是敏捷的“全团队”方法。有了敏捷,感觉对质量负责的不仅仅是测试人员或质量保证团队。我们不考虑“部门”,我们只考虑交付最佳产品所需的技能和资源。敏捷开发的重点是在一个时间框架内生产高质量的软件,使其对业务的价值最大化。这是整个团队的工作,而不仅仅是测试人员或指定的质量保证专业人员。敏捷团队中的每个人都会“被测试感染”。测试,从单元级别开始,推动编码,帮助团队了解应用程序应该如何工作,并让我们知道我们何时“完成”任务或故事。

One of the biggest differences in agile development versus traditional development is the agile “whole-team” approach. With agile, it’s not only the testers or a quality assurance team who feel responsible for quality. We don’t think of “departments,” we just think of the skills and resources we need to deliver the best possible product. The focus of agile development is producing high-quality software in a time frame that maximizes its value to the business. This is the job of the whole team, not just testers or designated quality assurance professionals. Everyone on an agile team gets “test-infected.” Tests, from the unit level on up, drive the coding, help the team learn how the application should work, and let us know when we’re “done” with a task or story.

敏捷团队必须具备生成高质量代码所需的所有技能,这些代码可提供组织所需的功能。虽然这可能意味着在团队中包括专家,例如专家测试人员,但它并不将特定任务限制为特定团队成员。任何任务都可能由任何团队成员或一对团队成员完成。这意味着团队负责各种测试任务,例如自动化测试和手动探索性测试。这也意味着整个团队不断地考虑设计可测试性代码。

An agile team must possess all the skills needed to produce quality code that delivers the features required by the organization. While this might mean including specialists on the team, such as expert testers, it doesn’t limit particular tasks to particular team members. Any task might be completed by any team member, or a pair of team members. This means that the team takes responsibility for all kinds of testing tasks, such as automating tests and manual exploratory testing. It also means that the whole team thinks constantly about designing code for testability.

全团队方法涉及持续协作。测试人员与程序员、客户团队和其他团队专家协作——不仅是为了测试任务,还有与测试相关的其他任务,例如构建基础设施和设计可测试性。图 1-5显示了开发人员与两个客户和一个测试人员(未图示)一起审查报告。

The whole-team approach involves constant collaboration. Testers collaborate with programmers, the customer team, and other team specialists—and not just for testing tasks, but other tasks related to testing, such as building infrastructure and designing for testability. Figure 1-5 shows a developer reviewing reports with two customers and a tester (not pictured).

图 1-5开发人员与客户讨论问题

Figure 1-5 A developer discusses an issue with customers

图像

全团队方法意味着每个人都对测试任务负责。这意味着团队成员拥有一系列技能和经验,可用于应对挑战,例如通过将示例转化为测试并转化为代码以使这些测试通过的可测试性设计。这些不同的观点只能意味着更好的测试和测试覆盖率。

The whole-team approach means everyone takes responsibility for testing tasks. It means team members have a range of skill sets and experience to employ in attacking challenges such as designing for testability by turning examples into tests and into code to make those tests pass. These diverse viewpoints can only mean better tests and test coverage.

最重要的是,在敏捷团队中,任何人都可以寻求和获得帮助。团队致力于提供尽可能高的团队商业价值,并且团队会尽一切努力来实现它。一些刚接触敏捷的人认为敏捷就是速度。事实是,一切都与质量有关——如果不是,我们会质疑它是否真的是一个“敏捷”团队。

Most importantly, on an agile team, anyone can ask for and receive help. The team commits to providing the highest possible business value as a team, and the team does whatever is needed to deliver it. Some folks who are new to agile perceive it as all about speed. The fact is, it’s all about quality—and if it’s not, we question whether it’s really an “agile” team.

你的情况很独特。这就是为什么您需要了解您的团队可能面临的潜在测试障碍以及如何应用敏捷价值观和原则来克服这些障碍。

Your situation is unique. That’s why you need to be aware of the potential testing obstacles your team might face and how you can apply agile values and principles to overcome them.

概括

Summary

了解测试人员在敏捷团队中执行的活动有助于您向自己的团队展示测试人员可以增加的价值。学习敏捷测试的核心实践将帮助您的团队交付令客户满意的软件。

Understanding the activities that testers perform on agile teams helps you show your own team the value that testers can add. Learning the core practices of agile testing will help your team deliver software that delights your customers.

在本章中,我们解释了使用术语“敏捷测试”时的含义。

In this chapter, we’ve explained what we mean when we use the term “agile testing.

图像我们展示了敏捷宣言如何与测试相关,它强调个人和交互、工作软件、客户协作和响应变化。

We showed how the Agile Manifesto relates to testing, with its emphasis on individuals and interactions, working software, customer collaboration, and responding to change.

图像我们为本书提供了一些背景信息,包括我们使用的一些其他术语,例如“测试人员”、“程序员”、“客户”和相关术语,以便我们可以使用共同语言。

We provided some context for this book, including some other terms we use such as “tester,” “programmer,” “customer,” and related terms so that we can speak a common language.

图像我们解释了专注于业务价值和交付客户所需质量的敏捷测试与专注于需求一致性的传统测试有何不同。

We explained how agile testing, with its focus on business value and delivering the quality customers require, is different from traditional testing, which focuses on conformance to requirements.

图像我们引入了“全团队”敏捷测试方法,这意味着参与交付软件的每个人都有责任交付高质量的软件。

We introduced the “whole-team” approach to agile testing, which means that everyone involved with delivering software is responsible for delivering high-quality software.

图像我们建议采取实用的方法,通过应用敏捷价值观和原则来克服在您的独特情况下出现的敏捷测试障碍。

We advised taking a practical approach by applying agile values and principles to overcome agile testing obstacles that arise in your unique situation.

第 2 章敏捷测试人员的十项原则

Chapter 2 Ten Principles for Agile Testers

图像

敏捷团队中的每个人都是测试人员。任何人都可以接手测试任务。如果那是真的,那么敏捷测试人员有什么特别之处呢?如果我将自己定义为敏捷团队的测试员,那究竟意味着什么?敏捷测试人员是否需要与传统团队中的测试人员不同的技能组合?是什么指导他们进行日常活动?

Everyone on an agile team is a tester. Anyone can pick up testing tasks. If that’s true, then what is special about an agile tester? If I define myself as a tester on an agile team, what does that really mean? Do agile testers need different skill sets than testers on traditional teams? What guides them in their daily activities?

在本章中,我们将讨论敏捷测试思维模式,展示敏捷价值观和原则如何指导测试,并概述测试人员如何为敏捷团队增加价值。

In this chapter, we talk about the agile testing mind-set, show how agile values and principles guide testing, and give an overview of how testers add value on agile teams.

什么是敏捷测试员?

What’s an Agile Tester?

我们这样定义敏捷测试人员:拥抱变化的专业测试人员,与技术人员和业务人员合作良好,理解使用测试记录需求和推动开发的概念。敏捷测试人员往往具有良好的技术技能,知道如何与其他人进行自动化测试,并且也是经验丰富的探索性测试人员。他们愿意了解客户的行为,以便更好地了解客户的软件需求。

We define an agile tester this way: a professional tester who embraces change, collaborates well with both technical and business people, and understands the concept of using tests to document requirements and drive development. Agile testers tend to have good technical skills, know how to collaborate with others to automate tests, and are also experienced exploratory testers. They’re willing to learn what customers do so that they can better understand the customers’ software requirements.

谁是敏捷测试员?她是推动敏捷测试的团队成员。我们认识许多敏捷测试人员,他们都是从其他专业开始的。开发人员被测试感染并扩展到单元测试之外。习惯于以敏捷方式工作的探索性测试人员被敏捷团队的想法所吸引。担任其他角色的专业人员,例如业务或职能分析师,可能具有相同的特征并做很多相同的工作。

Who’s an agile tester? She’s a team member who drives agile testing. We know many agile testers who started out in some other specialization. A developer becomes test-infected and branches out beyond unit testing. An exploratory tester, accustomed to working in an agile manner, is attracted to the idea of an agile team. Professionals in other roles, such as business or functional analysts, might share the same traits and do much of the same work.

能力固然重要,但态度更重要。珍妮特喜欢说:“没有态度,技巧就一文不值。” 在不得不为我们的敏捷团队雇佣大量测试人员之后,我们对此进行了很多思考,并与敏捷社区中的其他人进行了讨论。测试人员倾向于看到全局。他们更多地从用户或客户的角度来看待应用程序,这意味着他们通常以客户为中心。

Skills are important, but attitude counts more. Janet likes to say, “Without the attitude, the skill is nothing.” Having had to hire numerous testers for our agile teams, we’ve put a lot of thought into this and discussed it with others in the agile community. Testers tend to see the big picture. They look at the application more from a user or customer point of view, which means they’re generally customer-focused.

敏捷测试思维模式

The Agile Testing Mind-Set

是什么让团队“敏捷”?对我们来说,敏捷团队是一个不断专注于尽最大努力并交付尽可能最好的产品的团队。根据我们的经验,这涉及大量的纪律、学习、时间、实验和合作。它并不适合所有人,但它非常适合我们这些喜欢团队活力并专注于持续改进的人。

What makes a team “agile”? To us, an agile team is one that continually focuses on doing its best work and delivering the best possible product. In our experience, this involves a ton of discipline, learning, time, experimentation, and working together. It’s not for everyone, but it’s ideal for those of us who like the team dynamic and focus on continual improvement.

成功的项目是允许好人做好工作的结果。使某人在敏捷团队中成为成功的测试员的特征可能与在任何团队中成为高价值测试员的特征相同。

Successful projects are a result of good people allowed to do good work. The characteristics that make someone succeed as a tester on an agile team are probably the same characteristics that make a highly valued tester on any team.

敏捷测试人员不会将自己视为质量警察,保护她的客户免受不适当代码的影响。她准备好收集和共享信息,与客户或产品负责人一起工作,以帮助他们充分表达他们的需求,以便他们能够获得所需的功能,并向每个人提供项目进展的反馈。

An agile tester doesn’t see herself as a quality police officer, protecting her customers from inadequate code. She’s ready to gather and share information, to work with the customer or product owner in order to help them express their requirements adequately so that they can get the features they need, and to provide feedback on project progress to everyone.

敏捷测试人员,也许还有任何具有正确技能和思维方式的测试人员,都在不断寻找团队可以更好地生产高质量软件的方法。在个人层面上,这可能意味着参加本地用户组会议或圆桌会议以了解其他团队在做什么。它也意味着尝试新工具来帮助团队更好地指定、执行和自动化客户需求作为测试。

Agile testers, and maybe any tester with the right skills and mind-set, are continually looking for ways the team can do a better job of producing high-quality software. On a personal level, that might mean attending local user group meetings or roundtables to find out what other teams are doing. It also means trying out new tools to help the team do a better job of specifying, executing, and automating customer requirements as tests.

底线是敏捷测试人员,就像他们的敏捷队友一样,喜欢学习新技能和接受新挑战,他们不会局限于只解决测试问题。这不仅仅是测试人员的特征;我们在所有敏捷团队成员中都看到了这一点。敏捷测试人员帮助开发人员和客户团队解决可能出现的任何类型的问题。测试人员可以提供信息,帮助团队回顾并了解哪些有效,哪些无效。

The bottom line is that agile testers, like their agile teammates, enjoy learning new skills and taking on new challenges, and they don’t limit themselves to solving only testing issues. This isn’t just a trait of testers; we see it in all agile team members. Agile testers help the developer and customer teams address any kind of issue that might arise. Testers can provide information that helps the team look back and learn what’s working and what isn’t.

创造力、对想法的开放性、愿意承担任何任务或角色、关注客户以及对大局的持续看法只是敏捷测试思维模式的一些组成部分。优秀的测试人员对软件可能在何处以及如何失败以及如何追踪失败有直觉和理解。

Creativity, openness to ideas, willingness to take on any task or role, focus on the customer, and a constant view of the big picture are just some components of the agile testing mind-set. Good testers have an instinct and understanding for where and how software might fail, and how to track down failures.

测试人员可能在测试方面具有特殊的专业知识和经验,但优秀的敏捷测试人员不怕跳入设计讨论,提出有助于提高可测试性或创建更优雅的解决方案的建议。敏捷测试思维模式是一种以结果为导向、工匠般的、协作的、渴望学习的、热衷于及时交付业务价值的思维模式。

Testers might have special expertise and experience in testing, but a good agile tester isn’t afraid to jump into a design discussion with suggestions that will help testability or create a more elegant solution. An agile testing mind-set is one that is results-oriented, craftsman-like, collaborative, eager to learn, and passionate about delivering business value in a timely manner.

应用敏捷原则和价值观

Applying Agile Principles and Values

个人可以对项目的成功产生重大影响。我们期望一个拥有更多经验和更高技能成员的团队能够胜过一个缺乏天赋的团队。但是一个团队不仅仅是它的单个成员。敏捷价值观和原则提倡关注参与项目的人员以及他们如何互动和沟通。一个以敏捷价值观和原则指导自己的团队将比一个由人才组成的低效团队具有更高的团队士气和更快的速度。

Individuals can have a big impact on a project’s success. We’d expect a team with more experienced and higher-skilled members to outperform a less talented team. But a team is more than just its individual members. Agile values and principles promote a focus on the people involved in a project and how they interact and communicate. A team that guides itself with agile values and principles will have higher team morale and better velocity than a poorly functioning team of talented individuals.

我们在第一章开头介绍的敏捷宣言中的四个价值陈述显示的是偏好,而不是最后通牒,并且没有说明做什么或不做什么。敏捷宣言还包括一系列定义我们如何进行软件开发的原则。我们的敏捷“测试”原则列表部分源自这些原则。因为我们都来自极限编程文化,所以我们采纳了它的许多价值观和基本原则。我们还纳入了对我们的团队有效的指导方针和原则。您的团队自己的价值观和原则将在您选择实践和决定您希望如何工作时指导您。

The four value statements in the Agile Manifesto, which we presented at the start of the first chapter, show preferences, not ultimatums, and make no statements about what to do or not to do. The Agile Manifesto also includes a list of principles that define how we approach software development. Our list of agile “testing” principles is partially derived from those principles. Because we both come from the Extreme Programming culture, we’ve adopted many of its values and underlying principles. We’ve also incorporated guidelines and principles that have worked for our teams. Your team’s own values and principles will guide you as you choose practices and make decisions about how you want to work.

我们认为对于敏捷测试人员来说重要的原则是:

The principles we think are important for an agile tester are:

提供持续反馈

Provide Continuous Feedback

鉴于测试驱动敏捷项目,反馈在任何敏捷团队中都起着重要作用也就不足为奇了。测试人员作为“信息提供者”的传统角色使她对敏捷团队具有内在的价值。敏捷测试人员最重要的贡献之一是帮助产品所有者或客户以示例和测试的形式阐明每个故事的需求。然后,测试人员与队友一起工作,将这些需求转化为可执行的测试。测试人员、程序员和其他团队成员尽早并经常运行这些测试,以便他们不断得到有意义的反馈的指导。我们将在本书中花大量时间解释如何做到这一点。

Given that tests drive agile projects, it’s no surprise that feedback plays a big part in any agile team. The tester’s traditional role of “information provider” makes her inherently valuable to an agile team. One of the agile tester’s most important contributions is helping the product owner or customer articulate requirements for each story in the form of examples and tests. The tester then works together with teammates to turn those requirements into executable tests. Testers, programmers, and other team members work to run these tests early and often so they’re continually guided by meaningful feedback. We’ll spend a lot of time in this book explaining ways to do this.

当团队遇到障碍时,反馈是帮助消除障碍的一种方式。我们是否提供了一个不能完全满足客户期望的用户界面?让我们写一张任务卡,提醒我们在下一个 UI 故事的纸质原型上与客户合作。

When the team encounters obstacles, feedback is one way to help remove them. Did we deliver a user interface that didn’t quite meet customer expectations? Let’s write a task card reminding us to collaborate with the customer on paper prototypes of the next UI story.

管理层是否担心工作的进展情况?显示每天编写、运行和通过的测试的大图表。显示大图功能覆盖范围,例如测试矩阵。难以使构建稳定?Lisa 的团队显示了距离标记要发布的版本还有多少天,以便让每个人都专注于及时完成故事。在这成为一种习惯之后,他们就不再需要视觉提示了。

Is management worried about how work is progressing? Display a big visible chart of tests written, run, and passing every day. Display big-picture functionality coverage such as test matrices. Having trouble getting the build stable? Lisa’s team displayed the number of days remaining until time to tag the build for release in order to keep everyone focused on finishing stories in time. After that became a habit, they didn’t need the visual cue anymore.

为客户提供价值

Deliver Value to the Customer

敏捷开发是关于在提供客户最近优先考虑的功能的小版本中交付价值。这通常意味着限制范围。很容易被客户吸引团队对酷功能的渴望。任何人都可以质疑这些添加,但测试人员通常会认识到对故事的影响,因为他们需要考虑测试的影响。

Agile development is about delivering value in small releases that provide exactly the functionality that the customer has most recently prioritized. This usually means limiting scope. It’s easy to get caught up in the customer team’s desire for cool features. Anyone can question these additions, but a tester often recognizes the impact to the story, because they need to think about the testing repercussions.

丽莎的故事

Lisa’s Story

我们的产品负责人在每次迭代之前都会参加计划会议。然而,在迭代开始后,我们讨论了关于故事的更多细节以及如何测试它们,他经常提出一个在计划期间没有出现的想法,例如,“嗯,如果此报告的选择可能包括 X、Y 和 Z,并且也按 A 排序。” 一个无辜的请求会给故事增加很多复杂性。我经常带一个程序员来讨论这个添加是否可以在我们计划的故事范围内处理。如果没有,我们会要求产品负责人为下一次迭代写一张卡片。

Our product owner participates in planning meetings before each iteration. Nevertheless, after the iteration has started and we discuss more details about the stories and how to test them, he often brings up an idea that didn’t come out during the planning, such as, “Well, it would really be nice if the selection on this report could include X, Y, and Z and be sorted on A as well.” An innocent request can add a lot of complexity to a story. I often bring in one of the programmers to talk about whether this addition can be handled within the scope of the story we had planned. If not, we ask the product owner to write a card for the next iteration.

——丽莎

—Lisa

敏捷测试人员始终专注于大局。我们可以在此迭代中交付最关键的功能,并在以后添加。如果我们让新功能潜入,我们就有可能无法按时交付任何东西。如果我们过于关注边缘案例而错过了快乐道路上的核心功能,我们将无法提供业务所需的价值。

Agile testers stay focused on the big picture. We can deliver the most critical functionality in this iteration and add to it later. If we let new features creep in, we risk delivering nothing on time. If we get too caught up with edge cases and miss core functionality on the happy path, we won’t provide the value the business needs.

丽莎的故事

Lisa’s Story

为确保我们在每次迭代中交付一些价值,我们的团队会查看每个故事以确定必要功能的“关键路径”或“薄片”。我们首先完成这些任务,然后返回并充实其余功能。最坏的情况是只发布核心功能。这比什么都不提供或只提供半途而废的东西要好。

To ensure that we deliver some value in each iteration, our team looks at each story to identify the “critical path” or “thin slice” of necessary functionality. We complete those tasks first and then go back and flesh out the rest of the features. The worst-case scenario is that only the core functionality gets released. That’s better than delivering nothing or something that works only halfway.

——丽莎

—Lisa

敏捷测试人员采用与 Lisa 的故事中确定的方法相同的方法。虽然我们的一项技能是识别“快乐路径”之外的测试用例,但我们仍然需要从确保快乐路径有效开始。我们可以为快乐路径自动化测试,并在以后添加负面和边界测试。始终考虑什么能为客户增加最大价值,并了解您的背景。如果应用程序是安全关键的,则绝对需要添加负面测试。在估算过程中需要考虑测试时间,以确保在迭代中分配足够的时间来交付“安全”功能。

Agile testers take the same approach as that identified in Lisa’s story. While one of our skills is to identify test cases beyond the “happy path,” we still need to start by making sure the happy path works. We can automate tests for the happy path, and add negative and boundary tests later. Always consider what adds the most value to the customer, and understand your context. If an application is safety-critical, adding negative tests is absolutely required. The testing time needs to be considered during the estimation process to make sure that enough time is allotted in the iteration to deliver a “safe” feature.

实现面对面交流

Enable Face-to-Face Communication

没有良好的沟通,任何团队都无法运作良好。今天,当这么多团队分布在多个地理位置时,沟通是甚至更重要,更具挑战。敏捷测试人员应该寻找促进沟通的独特方式。这是做好她工作的一个关键方面。

No team works well without good communication. Today, when so many teams are distributed in multiple geographical locations, communication is even more vital and more of a challenge. The agile tester should look for unique ways to facilitate communication. It is a critical aspect to doing her job well.

珍妮特的故事

Janet’s Story

当我在一个团队工作时,我们遇到了一个真正的问题,即程序员与产品负责人交谈而将测试人员排除在讨论之外。他们经常在事后发现变化。部分问题是由于后勤问题,开发人员没有与测试人员坐在一起。另一个问题是历史。测试团队是新人,产品负责人习惯于直接去找程序员。

When I was working with one team, we had a real problem with programmers talking with the product owner and leaving the testers out of the discussion. They often found out about changes after the fact. Part of the problem was that the developers were not sitting with the testers due to logistical problems. Another problem was history. The test team was new, and the product owner was used to going straight to the programmers.

我把问题带到了团队,我们制定了规则。我们通过“三的力量”取得了巨大的成功。这意味着所有关于功能的讨论都需要程序员、测试人员和产品所有者。每个人都有责任确保每个小组始终有一名代表。如果有人看到两个人在说话,他们有权插话。没过多久,它就成了例行公事,没有人会考虑将测试人员排除在讨论之外。这对我们有用,因为团队购买了该解决方案。

I took the problem to the team, and we created a rule. We found great success with the “Power of Three.” This meant that all discussions about a feature needed a programmer, a tester, and the product owner. It was each person’s responsibility to make sure there was always a representative from each group. If someone saw two people talking, they had the right to butt into the conversation. It didn’t take very long before it was just routine and no one would consider leaving the tester out of a discussion. This worked for us because the team bought into the solution.

——珍妮特

—Janet

任何时候出现关于功能应该如何工作或界面应该是什么样子的问题时,测试人员都可以召集程序员和业务专家来讨论。测试人员永远不应该妨碍任何直接的客户与开发人员的沟通,但他们通常可以帮助确保沟通的发生。

Any time there is a question about how a feature should work or what an interface should look like, the tester can pull in a programmer and a business expert to talk about it. Testers should never get in the way of any direct customer-developer communication, but they can often help to make sure that communication happens.

敏捷测试人员从客户的角度看待每个故事或主题,但也了解与实现功能相关的技术方面和限制。他们可以帮助客户和开发人员实现共同语言。业务人员和软件人员经常说不同的语言。他们必须找到一些共同点才能成功合作。测试人员可以帮助他们开发一种共享语言、项目方言或团队行话。

Agile testers see each story or theme from the customer’s point of view but also understand technical aspects and limitations related to implementing features. They can help customers and developers achieve a common language. Business people and software people often speak different languages. They have to find some common ground in order to work together successfully. Testers can help them develop a shared language, a project dialect, or team jargon.

Brian Marick (2003) 建议我们使用示例来开发这种语言。当 Lisa 的团队在冲刺计划会议期间进行哲学讨论时,Lisa 向产品负责人询问示例或使用场景。测试人员可以通过更多示例鼓励白板讨论。这些有助于客户更清楚地设想他们的要求。它们还帮助开发人员生成设计良好的代码以满足这些要求。

Brian Marick (2003) recommends that we use examples to develop this language. When Lisa’s team digresses into a philosophical discussion during a sprint planning meeting, Lisa asks the product owner for an example or usage scenario. Testers can encourage whiteboard discussions to work through more examples. These help the customers envision their requirements more clearly. They also help the developers to produce well-designed code to meet those requirements.

面对面的交流无可替代。敏捷开发依赖于持续的协作。与其他敏捷团队成员一样,执行测试任务的人员将不断寻找客户和技术团队成员进行讨论和协作。当敏捷测试人员怀疑隐藏的假设或被误解的需求时,她会让客户和开发人员讨论它。如果不同建筑物或大陆的人们需要交谈,他们会寻找创造性的方式来取代面对面的实时对话。

Face-to-face communication has no substitute. Agile development depends on constant collaboration. Like other agile team members, the people doing testing tasks will continually seek out customer and technical team members to discuss and collaborate. When an agile tester suspects a hidden assumption or a misunderstood requirement, she’ll get a customer and a developer talking about it. If people in a different building or continent need to talk, they look for creative ways to replace face-to-face, real-time conversations.

有勇气

Have Courage

勇气是 XP 中的核心价值,测试自动化和持续集成等实践让团队践行这一价值。开发人员有勇气进行更改和重构代码,因为他们拥有自动回归套件的安全网。在本节中,我们将讨论过渡到敏捷团队时所需的情感勇气。

Courage is a core value in XP, and practices such as test automation and continuous integration allow the team to practice this value. The developers have the courage to make changes and refactor the code because they have the safety net of an automated regression suite. In this section, we talk about the emotional courage that is needed when transitioning to an agile team.

您是否曾在一个组织中工作,测试人员被困在自己的孤岛中,无法与业务利益相关者或技术团队的其他成员交谈?虽然您可能会抓住机会加入协作敏捷环境,但您可能会觉得不得不去询问客户示例,或者要求程序员帮助自动化测试或在每日站立会议中提出障碍,这让您感到不自在。

Have you worked in an organization where testers were stuck in their own silo, unable to talk to either business stakeholders or other members of the technical team? While you might jump at the chance to join a collaborative agile environment, you might feel uncomfortable having to go ask the customer for examples, or ask a programmer to help automate a test or bring up a roadblock during the daily stand-up.

当您第一次加入敏捷团队时,或者当您当前的团队第一次过渡到敏捷开发时,感到恐惧并列出需要回答的问题列表是很正常的。这么短的时间,我们要如何完成每个故事的测试任务?测试将如何“跟上”开发?你怎么知道多少测试就足够了?或者,您可能是一名功能测试经理或质量流程经理,您不清楚该角色在敏捷团队中的位置,而且没有人知道答案。敏捷测试人员需要勇气来找到这些问题的答案,但也有其他需要勇气的原因。

When you first join an agile team, or when your current team firsts transitions to agile development, it’s normal to experience fear and have a list of questions that need to be answered. How in the world are we going to be able to complete testing tasks for each story in such a short time? How will testing “keep up” with development? How do you know how much testing is enough? Or maybe you’re a functional testing manager or a quality process manager and it’s not clear to you where that role fits on an agile team, and nobody has the answers. Agile testers need courage to find the answers to those questions, but there are other reasons as well for having courage.

我们需要勇气让自己失败,知道至少我们会很快失败并能够从失败中吸取教训。在我们因为没有获得稳定的构建而导致迭代失败后,我们将开始考虑确保它不会再次发生的方法。

We need courage to let ourselves fail, knowing that at least we’ll fail fast and be able to learn from that failure. After we’ve blown an iteration because we didn’t get a stable build, we’ll start thinking of ways to ensure it doesn’t happen again.

我们需要勇气允许别人犯错,因为这是吸取教训的唯一途径。

We need courage to allow others to make mistakes, because that’s the only way to learn the lesson.

丽莎的故事

Lisa’s Story

我在一个项目中工作,敏捷教练坚持让我加入一个单独的测试团队(通常是一个团队!),他们的工作不包括在程序员的跟踪和速度中。我不得不继续尝试这个。在发布因为测试未完成而遇到麻烦后,我问教练我们是否可以按照我的方式尝试一两次迭代。整个团队的方法效果更好。每个故事都经过测试并在迭代结束时“完成”,客户对结果更加满意。

I worked on a project where the agile coach insisted that I be on a separate testing team (often a team of one!) whose work wasn’t included in the programmers’ tracking and velocity. I had to just go along and try this. After the release ran into trouble because testing wasn’t finished, I asked the coach if we could try things my way for an iteration or two. The whole-team approach worked much better. Each story was tested and “done” by the end of the iteration, and the customers were much happier with the results.

——丽莎

—Lisa

我们需要勇气寻求帮助,尤其是当可以提供帮助的人看起来非常忙碌而且压力很大时。走出你的旧筒仓并加入一个团队来为成功或失败负责需要勇气。提出问题或指出您认为的缺陷需要勇气,即使在一个由敏捷价值观和原则支持的团队中也是如此。不要害怕!敏捷团队是开放的,通常接受新想法。

We need courage to ask for help, especially when the person who could provide that help looks pretty busy and stressed-out himself. Climbing out of your old silo and joining in a team responsibility for success or failure takes courage. Asking a question or pointing out what you think is a flaw requires courage, even in a team supported by agile values and principles. Don’t be afraid! Agile teams are open and generally accepting of new ideas.

把事情简单化

Keep It Simple

Kent Beck 的Extreme Programming Explained建议我们做可能可行的最简单的事情。这并不意味着您尝试的第一件事实际上会起作用,但它应该很简单。

Kent Beck’s Extreme Programming Explained advised us to do the simplest thing that could possibly work. That doesn’t mean the first thing you try will actually work, but it ought to be simple.

敏捷测试人员及其团队面临的挑战不仅是要生成最简单的软件实现,还要采取简单的方法来确保软件满足客户需求。这并不意味着团队不应该花一些时间来分析主题和故事并思考适当的架构和设计。这确实意味着当他们的需求可能有点复杂并且更简单的解决方案将提供相同的价值时,团队可能需要将其推回到团队的业务方面。

Agile testers and their teams are challenged to not only produce the simplest possible software implementation but to take a simple approach to ensuring that software meets the customer requirements. This doesn’t mean that the team shouldn’t take some time to analyze themes and stories and think through the appropriate architecture and design. It does mean that the team might need to push back to the business side of the team when their requirements might be a bit elaborate and a simpler solution will deliver the same value.

我们中的一些人在软件组织工作,作为测试人员和质量保证人员,我们被要求制定质量标准。我们认为这是倒退的,因为这取决于客户团队来决定他们想要支付什么样的质量水平。测试人员和其他团队成员应该向客户提供信息并帮助他们考虑质量的所有方面,包括性能和安全性等非功能性需求。最终决定取决于客户。团队可以通过采取简单的、循序渐进的方法来帮助客户做出正确的决定。敏捷测试手段进行尽可能简单的测试以验证某项功能是否存在或是否满足客户的质量标准(例如,性能)。

Some of us worked in software organizations where we, as testers and quality assurance staff, were asked to set quality standards. We believe this is backwards, because it’s up to the customer team to decide what level of quality they want to pay for. Testers and other team members should provide information to customers and help them consider all aspects of quality, including nonfunctional requirements such as performance and security. The ultimate decisions are up to the customer. The team can help the customer make good decisions by its taking a simple, step-by-step approach to its work. Agile testing means doing the simplest tests possible to verify that a piece of functionality exists or that the customer’s quality standard (e.g., performance) has been met.

简单并不意味着容易。对于测试人员来说,这意味着使用我们能找到的最轻量级的工具和技术来测试“刚好足够”,以完成这项工作。工具可以像电子表格或清单一样简单。我们需要自动化回归测试,但我们应该将它们降低到尽可能低的级别,以鼓励快速反馈。即使是简单的冒烟测试也可能足以实现面向业务的测试自动化。

Simple doesn’t mean easy. For testers, it means testing “just enough” with the lightest-weight tools and techniques we can find that will do the job. Tools can be as simple as a spreadsheet or a checklist. We need to automate regression tests, but we should push them down to the lowest level possible in order to encourage fast feedback. Even simple smoke tests might be enough for business-facing test automation.

探索性测试可用于了解您的应用程序并找出难以发现的错误,但要从基础知识、时间限制的旁路测试和评估边缘案例的进展情况开始。简单有助于我们将注意力集中在风险、投资回报和改进最痛苦的领域。

Exploratory testing can be used to learn about your application and ferret out hard-to-find bugs, but start with the basics, time-boxing side trips and evaluating how far to go with edge cases. Simplicity helps us keep our focus on risk, return on investment, and improving in the areas of greatest pain.

第 IV 部分,“测试自动化”,解释了如何构建“可行的”测试自动化策略。

Part IV, “Test Automation,” explains how to build a “doable” test automation strategy.

实践持续改进

Practice Continuous Improvement

寻找方法来做得更好是敏捷测试人员心态的一部分。当然,整个团队都应该这样想,因为敏捷的中心核心是团队总是努力把工作做得更好。测试人员参与团队回顾,评估哪些运作良好,哪些需要添加或调整。测试人员将测试问题提交给整个团队来解决。团队通过使用过程改进实践(例如回顾和障碍积压)在测试和所有其他领域取得了最大的改进。一些改进想法可能会成为任务卡。对于更大的问题,团队一次只关注一两个问题,以确保他们解决了真正的问题,而不仅仅是表面现象。

Looking for ways to do a better job is part of an agile tester’s mind-set. Of course, the whole team should be thinking this way, because the central core of agile is that the team always tries to do better work. Testers participate in team retrospectives, evaluating what’s working well and what needs to be added or tweaked. Testers bring testing issues up for the whole team to address. Teams have achieved their greatest improvements in testing and all other areas through the use of process improvement practices such as retrospectives and impediment backlogs. Some improvement ideas might become task cards. For larger problems, teams focus on one or two issues at a time to make sure they solve the real problem and not just the symptom.

敏捷测试人员和他们的团队总是在寻找可以帮助他们增加更多价值或获得更好的客户投资回报的工具、技能或实践。敏捷开发的短迭代使得在几次迭代中尝试新事物变得更容易,并且看看它是否值得长期采用。

Agile testers and their teams are always on the lookout for tools, skills, or practices that might help them add more value or get a better return on the customer’s investment. The short iterations of agile development make it easier to try something new for a few iterations and see whether it’s worth adopting for the long term.

学习新技能和专业成长对敏捷测试人员来说很重要。他们利用许多可用的免费资源来提高他们的专业技能,例如探索性测试。他们参加各种会议、加入邮件列表、阅读文章、博客和书籍以获得新想法。他们寻找自动化(或从同事那里获得帮助以自动化)平凡或重复性任务的方法,以便他们有更多时间贡献他们宝贵的专业知识。

Learning new skills and growing professionally are important to agile testers. They take advantage of the many available free resources to improve their specialized skills, such as exploratory testing. They go to meetings and conferences, join mailing lists, and read articles, blogs, and books to get new ideas. They look for ways to automate (or get help from their coworkers to automate) mundane or repetitive tasks so they have more time to contribute their valuable expertise.

Pierre Veragren 是 iLevel by Weyerhaeuser 的 SQA 负责人,他发现了我们自己在敏捷团队中经常看到的一种品质:“AADD”,即敏捷注意力缺陷障碍。任何不能快速学习的东西都可能被认为是无用的。敏捷团队成员寻求投资回报,如果他们没有很快看到回报,他们就会继续前进。当您每两周甚至更频繁地交付生产就绪软件时,这并不是一个负面特征。

Pierre Veragren, an SQA Lead at iLevel by Weyerhaeuser, identified a quality we often see in agile teams ourselves: “AADD,” Agile Attention Deficit Disorder. Anything not learned quickly might be deemed useless. Agile team members look for return on investment, and if they don’t see it quickly, they move on. This isn’t a negative characteristic when you’re delivering production-ready software every two weeks or even more often.

回顾是一种关键的敏捷实践,它可以让团队利用昨天的经验在明天做得更好。敏捷测试人员利用这个机会提出与测试相关的问题,并要求团队集思广益解决这些问题。这是团队向自己提供反馈以持续改进的一种方式。

Retrospectives are a key agile practice that lets the team use yesterday’s experience to do a better job tomorrow. Agile testers use this opportunity to raise testing-related issues and ask the team to brainstorm ways to address them. This is a way for the team to provide feedback to itself for continual improvement.

丽莎的故事

Lisa’s Story

我们的团队使用回顾获得了巨大的收益,但我们觉得我们需要一些新的东西来帮助我们专注于做得更好。我建议保留一个“障碍积压”的项目,这些项目使我们无法像我们希望的那样富有成效。我在障碍积压中写的第一件事是我们测试环境的缓慢响应时间。我们的系统管理员搜集了几台便宜的机器,并将它们变成了新的、更快的服务器,用于我们的测试环境。我们的 DBA 分析了测试数据库性能,发现单磁盘系统是障碍,我们的经理同意安装 RAID 以更好地访问磁盘。很快我们就能够更快地部署构建和进行探索性测试。

Our team had used retrospectives to great benefit, but we felt we needed something new to help us focus on doing a better job. I suggested keeping an “impediment backlog” of items that were keeping us from being as productive as we’d like to be. The first thing I wrote in the impediment backlog was our test environment’s slow response time. Our system administrator scrounged a couple of bargain machines and turned them into new, faster servers for our test environments. Our DBA analyzed the test database performance, found that the one-disk system was the impediment, and our manager gave the go-ahead to install a RAID for better disk access. Soon we were able to deploy builds and conduct our exploratory testing much faster.

——丽莎

—Lisa

我们将在第 19 章总结迭代”中更多地讨论回顾以及它们如何帮助您的团队实践持续改进。

We’ll talk more about retrospectives and how they can help your team practice continuous improvement in Chapter 19, “Wrap Up the Iteration.”

应对变化

Respond to Change

当我们在瀑布环境中工作时,我们习惯于说,“对不起,我们现在不能做这个改变;要求被冻结。我们必须把它放在第一个补丁版本中。” 这让客户感到沮丧,因为他们意识到他们没有很好地预先定义所有需求。

When we worked in a waterfall environment, we got used to saying, “Sorry, we can’t make this change now; the requirements are frozen. We’ll have to put that in the first patch release.” It was frustrating for customers because they realized that they didn’t do a great job on defining all their requirements up front.

在为期两周的敏捷迭代中,我们可能不得不说,“好吧,为此写一张卡片,我们将在下一次迭代或下一次发布中完成,”但客户知道他们可以在需要时获得更改,因为他们控制优先级。

In a two-week agile iteration, we might have to say, “OK, write a card for that and we’ll do it in the next iteration or next release,” but customers know they can get their change when they want it because they control the priority.

响应变化是敏捷实践者的一个关键价值,但我们发现它是测试人员最难理解的概念之一。稳定性是测试人员所渴望的,这样他们就可以说,“我已经测试过了;我已经测试过了;” 完成。” 不断变化的需求是测试人员的噩梦。但是,作为敏捷测试人员,我们必须欢迎变化。周三,我们可能会开始故事 A 和 B然后下周五 C。到周五,客户可能已经重新确定优先级,现在想要故事 A、X 和 Y。只要我们继续与客户交谈,我们就可以处理这样的变化,因为我们与团队其他成员的工作节奏相同.

Responding to change is a key value for agile practitioners, but we’ve found that it’s one of the most difficult concepts for testers. Stability is what testers crave so that they can say, “I’ve tested that; it’s done.” Continuously changing requirements are a tester’s nightmare. However, as agile testers, we have to welcome change. On Wednesday, we might expect to start stories A and B and then C the next Friday. By Friday, the customer could have re-prioritized and now wants stories A, X, and Y. As long as we keep talking to the customer, we can handle changes like that because we are working at the same pace with the rest of team.

一些敏捷团队尝试在下一次迭代之前做好准备,可能是通过编写高级测试用例、捕获业务满意度条件或记录示例。这是一项棘手的工作,如果重新确定故事的优先级或进行重大更改,可能会导致时间浪费。然而,分布式团队尤其需要额外的反馈周期来为迭代做好准备。

Some agile teams try to prepare in advance of the next iteration, perhaps by writing high-level test cases, capturing business satisfaction conditions, or documenting examples. It’s a tricky business that might result in wasted time if stories are re-prioritized or greatly changed. However, distributed teams in particular need extra feedback cycles to get ready for the iteration.

丽莎的故事

Lisa’s Story

我们的远程团队成员曾经是我们的现场经理。他是帮助企业编写故事和确定故事优先级的关键人物。他对代码和业务都有深入的了解,这有助于他针对业务需求提出创造性的解决方案。当他搬到印度时,我们想方设法保留他的专业知识。会议安排在他可以参加的时间,他定期与产品负责人进行电话会议,讨论即将发生的故事。我们不得不从索引卡等低科技工具转向我们都可以使用的在线工具。

Our remote team member used to be our on-site manager. He’s a key player in helping the business write and prioritize stories. He has in-depth knowledge of both the code and the business, which helps him come up with creative solutions to business needs. When he moved to India, we looked for ways to retain the benefit of his expertise. Meetings are scheduled at times when he can participate, and he has regular conference calls with the product owner to talk about upcoming stories. We’ve had to switch from low-tech tools such as index cards to online tools that we can all use.

因为团队愿意改变我们的工作方式,并寻找有助于让他了解持续变化的工具,所以我们能够保留他的专业知识带来的好处。

Because the team was willing to make changes in the way we worked, and looked for tools that helped keep him in the loop with ongoing changes, we were able to retain the benefit of his expertise.

——丽莎

—Lisa

有些团队有分析师,他们可以花更多时间与业务专家一起做一些预先计划。每个团队都必须在提前集思广益解决方案和在每次迭代的第一天从头开始之间取得平衡。敏捷测试人员顺应潮流并与团队合作以适应变化。

Some teams have analysts who can spend more time with the business experts to do some advance planning. Each team has to strike a balance between brainstorming solutions ahead of time and starting from scratch on the first day of each iteration. Agile testers go with the flow and work with the team to accommodate changes.

自动化测试是解决方案的关键之一。我们可以肯定地知道一件事:没有任何敏捷团队仅通过手动测试就能成功。我们需要强大的自动化,以便在使其有价值的时间范围内交付业务价值。

Automated testing is one key to the solution. One thing we know for sure: No agile team will succeed doing only manual testing. We need robust automation in order to deliver business value in a time frame that makes it valuable.

自组织

Self-Organize

敏捷测试人员是自组织敏捷团队的一部分。团队文化灌输了敏捷测试理念。当程序员、系统管理员、分析师、数据库专家和客户团队不断思考测试和测试自动化时,测试人员就会享受到全新的视角。自动化测试很难,但当你拥有整个团队时,它会容易得多一起工作。当您有具有多种技能和多种观点的人来攻击它时,任何测试问题都更容易解决。

The agile tester is part of a self-organizing agile team. The team culture imbues the agile testing philosophy. When programmers, system administrators, analysts, database experts, and the customer team think continually about testing and test automation, testers enjoy a whole new perspective. Automating tests is hard, but it is much easier when you have the whole team working together. Any testing issue is easier to address when you have people with multiple skill sets and multiple perspectives attacking it.

丽莎的故事

Lisa’s Story

我的团队是自组织团队的一个很好的例子。当我们实施 Scrum 时,我们有一个有缺陷的遗留系统并且没有自动化测试。对代码进行任何更改充其量是有风险的。我们的经理可能对这个问题有一些很好的解决方案,但他没有提出建议。相反,我们探索了这些问题并提出了一个计划。

My team is a good example of a self-organizing team. When we implemented Scrum, we had a buggy legacy system and no automated tests. Making any changes to the code was risky at best. Our manager probably had some excellent solutions to the problem, but he didn’t suggest them. Instead, we explored the issues and came up with a plan.

程序员将开始使用测试驱动开发在一个新的、可测试的架构中实现新故事。测试人员将编写手动回归测试脚本,整个团队(程序员、测试人员、系统管理员和 DBA)将在每次迭代的最后两天执行这些脚本。测试人员(当时,这意味着我)将通过用户界面在自动回归冒烟测试套件上工作。最终,新代码的架构将使我们能够使用 FitNesse 等工具自动进行功能测试。

The programmers would start implementing new stories in a new, testable architecture, using test-driven development. The testers would write manual regression test scripts, and the entire team—programmers, testers, the system administrator, and the DBA—would execute them on the last two days of every iteration. The testers (at the time, this meant me) would work on an automated regression smoke test suite through the user interface. Eventually, the architecture of the new code would let us automate functional tests with a tool such as FitNesse.

我们逐步实施了这个计划,在每次迭代中改进了我们的方法。使用团队中每个成员的技能比我自己决定自动化策略要好得多。

We implemented this plan in baby steps, refining our approach in each iteration. Using the skills of every member of the team was a much better approach than my going off and deciding the automation strategy on my own.

——丽莎

—Lisa

当一个敏捷团队面临一个大问题时,也许是生产停滞或构建失败,这是每个人的问题。最高优先级的问题是整个团队要解决的问题。团队成员立即讨论问题并决定如何解决以及由谁来解决。

When an agile team faces a big problem, perhaps a production showstopper or a broken build, it’s everyone’s problem. The highest-priority issues are problems for the whole team to solve. Team members discuss the issue right away and decide how to and who will fix it.

毫无疑问,Lisa 的经理可以要求团队采用这种方法来解决其自动化问题,但团队自己可以提出最可行的计划。当团队创建自己的方法并致力于它时,其成员就会对测试采取新的态度。

There’s no doubt that Lisa’s manager could have mandated that the team take this approach to solving its automation problems, but the team itself can come up with the most workable plan. When the team creates its own approach and commits to it, its members adopt a new attitude toward testing.

以人为本

Focus on People

当好人被允许做他们最好的工作时,项目就会成功。敏捷价值观和原则的创建旨在实现个人和团队的成功。敏捷团队成员应该感到安全,不必担心因错误而受到指责或失去工作。敏捷团队成员相互尊重并认可个人成就。敏捷团队中的每个人都应该有机会成长和发展他们的技能。敏捷团队以可持续的步伐工作,使他们能够遵循纪律严明的做法并保持全新的视角。正如敏捷宣言所述,我们重视个人和交互胜过流程和工具。

Projects succeed when good people are allowed to do their best work. Agile values and principles were created with the aim of enabling individual and team success. Agile team members should feel safe and not have to worry about being blamed for mistakes or losing their jobs. Agile team members respect each other and recognize individual accomplishments. Everyone on an agile team should have opportunities to grow and develop their skills. Agile teams work at a sustainable pace that lets them follow disciplined practices and keep a fresh perspective. As the Agile Manifesto states, we value individuals and interactions over processes and tools.

在软件开发的历史中,测试人员并不总是享有与开发团队中其他角色同等的地位。有些人将测试人员视为失败的程序员或软件开发领域的二等公民。懒得学习新技能和专业成长的测试人员会导致人们认为测试是一项低技能工作。甚至连“测试员”这个词也被避免了,优先使用“质量保证工程师”或“质量分析师”等职位以及“QA 部门”等团队名称。

In the history of software development, testers haven’t always enjoyed parity with other roles on the development team. Some people saw testers as failed programmers or second-class citizens in the world of software development. Testers who don’t bother to learn new skills and grow professionally contribute to the perception that testing is low-skilled work. Even the term “tester” has been avoided, with job titles such as “Quality Assurance Engineer” or “Quality Analyst” and team names such as “QA Department” given preference.

坚持真正敏捷哲学的敏捷团队给予所有团队成员同等的权重。敏捷测试人员知道他们为他们的团队贡献了独特的价值,而开发团队发现当他们的团队包括具有特定测试技能和背景的人时,他们会更加成功。例如,熟练的探索性测试人员可能会发现系统中无法通过自动功能测试检测到的问题。具有深厚测试经验的人可能会提出重要的问题,而这些问题对于没有测试经验的团队成员来说是不会想到的。测试知识是任何团队交付价值能力的组成部分之一。

Agile teams that adhere to the true agile philosophy give all team members equal weight. Agile testers know they contribute unique value to their teams, and development teams have found they are more successful when their team includes people with specific testing skills and background. For example, a skilled exploratory tester may discover issues in the system that couldn’t be detected by automated functional tests. Someone with deep testing experience might ask important questions that didn’t occur to team members without testing experience. Testing knowledge is one component of any team’s ability to deliver value.

享受

Enjoy

在一个每个人都协作的团队中工作,你从头到尾参与项目,业务利益相关者与开发团队一起工作,整个团队负责质量和测试,在我们看来,这简直就是测试者的乌托邦。我们并不是唯一相信每个人都应该在工作中找到乐趣的人。敏捷开发奖励敏捷测试人员对其工作的热情。

Working on a team where everyone collaborates, where you are engaged in the project from start to finish, where business stakeholders work together with the development team, where the whole team takes responsibility for quality and testing, in our opinion, is nothing short of a tester’s Utopia. We’re not alone in believing that everyone should find joy in their work. Agile development rewards the agile tester’s passion for her work.

我们作为敏捷测试人员的工作特别令人满意,因为我们的观点和技能让我们能够为我们的团队增加真正的价值。在下一节中,我们将探讨如何。

Our jobs as agile testers are particularly satisfying because our viewpoint and skills let us add real value to our teams. In the next section, we’ll explore how.

增加价值

Adding Value

这些原则给团队带来了什么?它们共同带来了商业价值。在敏捷开发中,整个团队负责交付高质量的软件,让客户满意并使业务更有利可图。这反过来又为企业带来了新的优势。

What do these principles bring to the team? Together, they bring business value. In agile development, the whole team takes responsibility for delivering high-quality software that delights customers and makes the business more profitable. This, in turn, brings new advantages for the business.

团队成员身兼数职,敏捷开发倾向于避免按专业对人进行分类。即使是短迭代和频繁发布,也很容易在客户团队的期望与团队交付的内容之间形成差距。使用测试来推动开发有助于防止这种情况发生,但您仍然需要正确的测试。

Team members wear many hats, and agile development tends to avoid classifying people by specialty. Even with short iterations and frequent releases, it’s easy to develop a gap between what the customer team expects and what the team delivers. Using tests to drive development helps to prevent this, but you still need the right tests.

敏捷测试人员不仅从将接受解决方案的利益相关者的角度考虑系统,而且还掌握了开发团队面临的技术约束和实施细节。程序员专注于让事情运转起来。如果他们按照正确的要求进行编码,客户会很高兴。不幸的是,客户通常不善于表达他们的需求。用错误的测试推动开发不会产生预期的结果。敏捷测试人员尽早并经常向客户和开发人员提出问题,并帮助将答案塑造成正确的测试。

Agile testers not only think about the system from the viewpoint of stakeholders who will live with the solution but they also have a grasp of technical constraints and implementation details that face the development team. Programmers focus on making things work. If they’re coding to the right requirements, customers will be happy. Unfortunately, customers aren’t generally good at articulating their requirements. Driving development with the wrong tests won’t deliver the desired outcome. Agile testers ask questions of both customers and developers early and often, and help shape the answers into the right tests.

与传统瀑布项目的测试人员相比,敏捷测试人员采用更加集成、面向团队的方法。他们使自己的技能和经验适应团队和项目。将程序员视为对手,或者坐等工作交给她,或者期望花更多时间计划而不是做的测试人员可能会坚持她在传统项目中学到的技能,并且不会在敏捷团队中待太久.

Agile testers take a much more integrated, team-oriented approach than testers on traditional waterfall projects. They adapt their skills and experience to the team and project. A tester who views programmers as adversaries, or sits and waits for work to come to her, or expects to spend more time planning than doing, is likely to cling to skills she learned on traditional projects and won’t last long on an agile team.

危险:你不是“真正”的团队成员

Peril: You’re Not “Really” Part of the Team

如果您是一名测试人员,并且没有被邀请参加计划会议、站立会议或设计会议,那么您可能处于这样一种情况,即测试人员在某种程度上被视为与开发团队不同。如果您被邀请参加这些会议,但您没有发言,那么您可能会造成一种感觉,即您并不是团队的一员。如果业务专家自己编写故事和定义需求,那么您就不是作为敏捷团队成员的测试人员参与的。

If you’re a tester, and you’re not invited to attend planning sessions, stand-ups, or design meetings, you might be in a situation where testers are viewed as somehow apart from the development team. If you are invited to these meetings but you’re not speaking up, then you’re probably creating a perception that you aren’t really part of the team. If business experts are writing stories and defining requirements all by themselves, you aren’t participating as a tester who’s a member of an agile team.

如果这是你的情况,你的团队就处于危险之中。隐藏的假设可能直到发布周期的后期才被发现。一个故事对系统其他部分的连锁反应直到为时已晚才被发现。团队没有充分利用每个团队成员的技能,因此无法生产出最好的软件。沟通可能会中断,并且很难跟上程序员和客户的工作。团队冒着以不健康的方式在开发人员和测试人员之间划分的风险,而且开发团队更有可能与客户团队隔离开来。

If this is your situation, your team is at risk. Hidden assumptions are likely to go undetected until late in the release cycle. Ripple effects of a story on other parts of the system aren’t identified until it’s too late. The team isn’t making the best use of every team member’s skills, so it’s not going to be able to produce the best possible software. Communication might break down, and it’ll be hard to keep up with what the programmers and customers are doing. The team risks being divided in an unhealthy way between developers and testers, and there’s more potential that the development team will become isolated from the customer team.

你怎样才能避免这种危险呢?看看您是否可以安排在开发商附近。如果你不能,至少来他们的区域交谈和配对测试。请他们向您展示他们在做什么。请他们查看您编写的测试用例。如果没有其他人邀请您,请邀请自己参加会议。通过测试和提供反馈让自己变得有用,并成为团队的必需品。

How can you avoid this peril? See if you can arrange to be located near the developers. If you can’t, at least come to their area to talk and pair test. Ask them to show you what they’re working on. Ask them to look at the test cases you’ve written. Invite yourself to meetings if nobody else has invited you. Make yourself useful by testing and providing feedback, and become a necessity to the team.

帮助客户开发他们的故事和验收测试。推动“整个团队”的态度,并要求团队致力于测试问题。如果您的团队在适应敏捷开发方面遇到困难,建议在一两次迭代中尝试一些新想法。建议采用“三的力量”规则来促进良好的沟通。使用本书中的信息表明测试人员可以帮助敏捷团队取得超越他们最疯狂期望的成功。

Help customers develop their stories and acceptance tests. Push the “whole team” attitude, and ask the team to work on testing problems. If your team is having trouble adapting to agile development, suggest experimenting with some new ideas for an iteration or two. Propose adopting the “Power of Three” rule to promote good communication. Use the information in this book to show that testers can help agile teams succeed beyond their wildest expectations.

在故事估算和计划会议期间,敏捷测试人员从多个角度审视每个特性:业务、最终用户、生产支持和程序员。他们考虑企业面临的问题以及软件如何解决这些问题。他们提出的问题会推翻客户和开发团队所做的假设。在每次迭代开始时,他们帮助确保客户提供明确的需求和示例,并帮助开发团队将这些转化为测试。测试推动开发,测试结果提供团队进度的反馈。测试人员帮助提出问题,以便不会忽视任何测试;它不仅仅是功能测试。客户并不总是知道他们应该提及他们的性能和可靠性需求或安全问题,但测试人员会考虑询问这些。测试人员还使测试方法和工具尽可能简单和轻便。在迭代结束时,测试人员验证是否已完成最低限度的测试。

During story estimating and planning sessions, agile testers look at each feature from multiple perspectives: business, end user, production support, and programmer. They consider the problems faced by the business and how the software might address them. They raise questions that flush out assumptions made by the customer and developer teams. At the start of each iteration, they help to make sure the customer provides clear requirements and examples, and they help the development team turn those into tests. The tests drive development, and test results provide feedback on the team’s progress. Testers help to raise issues so that no testing is overlooked; it’s more than functional testing. Customers don’t always know that they should mention their performance and reliability needs or security concerns, but testers think to ask about those. Testers also keep the testing approach and tools as simple and lightweight as possible. By the end of the iteration, testers verify that the minimum testing was completed.

敏捷团队中角色之间的界限是模糊的。其他团队成员可能擅长测试人员执行的相同活动。例如,分析师和程序员也编写面向业务的测试。只要执行了所有测试活动,敏捷团队就不一定需要主要将自己标识为测试人员的成员。然而,我们发现团队受益于专业测试人员开发的技能。我们讨论的敏捷原则和价值观将帮助任何团队做好测试和交付价值的工作。

Lines between roles on an agile team are blurred. Other team members might be skilled at the same activities that testers perform. For example, analysts and programmers also write business-facing tests. As long as all testing activities are performed, an agile team doesn’t necessarily require members who identify themselves primarily as testers. However, we have found that teams benefit from the skills that professional testers have developed. The agile principles and values we’ve discussed will help any team do a good job of testing and delivering value.

概括

Summary

在本章中,我们介绍了敏捷测试人员的原则以及我们认为敏捷测试人员需要具备的价值观,以便有效地为敏捷团队做出贡献。

In this chapter, we covered principles for agile testers and the values we think an agile tester needs to possess in order to contribute effectively to an agile team.

图像“敏捷测试思维模式”是以客户为中心、以结果为导向、工匠般的、协作的、创造性的、渴望学习的,并且热衷于及时交付业务价值。

An “agile testing mind-set” is customer-focused, results-oriented, craftsman-like, collaborative, creative, eager to learn, and passionate about delivering business value in a timely manner.

图像态度很重要,它模糊了测试人员、程序员和敏捷团队中其他角色之间的界限。

Attitude is important, and it blurs the lines between testers, programmers, and other roles on an agile team.

图像 敏捷测试人员应用敏捷的价值观和原则,如反馈、沟通、勇气、简单、享受和交付价值,以帮助团队识别和交付每个故事的客户需求。

Agile testers apply agile values and principles such as feedback, communication, courage, simplicity, enjoyment, and delivering value in order to help the team identify and deliver the customer requirements for each story.

图像敏捷测试人员以其独特的观点和面向团队的方法为他们的团队和组织增加价值。

Agile testers add value to their teams and their organizations with their unique viewpoint and team-oriented approach.

第二部分组织挑战

Part II Organizational Challenges

当软件开发组织实施敏捷开发时,测试或 QA 团队通常需要最长的时间才能完成过渡。独立的 QA 团队已经在许多组织中根深蒂固。当他们开始适应新的敏捷组织时,他们会遇到难以接受的文化差异。在第 II 部分中,我们讨论了引入变更以及您在向敏捷过渡时可能遇到的一些障碍。培训是进行转型的组织的重要组成部分,但往往被遗忘。也很难看出审计和流程改进框架等现有流程将如何在敏捷环境中发挥作用。从一个独立的 QA 团队到一个集成的敏捷团队是一个巨大的变化。

When software development organizations implement agile development, the testing or QA team often takes the longest to make the transition. Independent QA teams have become entrenched in many organizations. When they start to adapt to a new agile organization, they encounter cultural differences that are difficult for them to accept. In Part II, we talk about introducing change and some of the barriers you might encounter when transitioning to agile. Training is a big part of what organizations making the transition need, and it’s often forgotten. It’s also hard to see how existing processes such as audits and process improvement frameworks will work in the agile environment. Going from an independent QA team to an integrated agile team is a huge change.

第 4 章团队后勤”讨论了团队结构,例如测试人员在团队中的实际位置,以及关于测试人员与开发人员比率的永无止境的问题。我们还将讨论招聘测试人员以及在成功的敏捷测试人员中寻找什么。

Chapter 4, “Team Logistics,” talks about the team structure, such as where a tester actually fits into the team, and the never-ending question about tester-developer ratio. We’ll also talk about hiring testers and what to look for in a successful agile tester.

传统的测试活动,例如记录错误、跟踪指标和编写测试计划,可能看起来不太适合敏捷项目。我们介绍了一些可能需要特别注意和注意的典型流程,并讨论了如何调整现有的质量流程。

Traditional testing activities, such as logging bugs, keeping track of metrics, and writing test plans, might not seem like a good fit in an agile project. We introduce some of the typical processes that might need special care and attention and discuss how to adapt existing quality processes.

您可以期望找到一些方法,让习惯于传统瀑布式开发环境的测试人员和测试团队可以改变他们的组织结构和文化,从而从敏捷开发中受益并为敏捷开发增加价值。

You can expect to find ways that testers and test teams accustomed to a traditional waterfall type of development environment can change their organizational structure and culture to benefit from and add value to agile development.

第三章文化挑战

Chapter 3 Cultural Challenges

图像

许多组织影响都会影响项目,无论它使用敏捷还是传统的分阶段或门控方法。组织和团队文化会阻碍向敏捷方法的平稳过渡。在本章中,我们将讨论可以直接影响测试人员在敏捷团队中的角色的因素。

Many organizational influences can impact a project, whether it uses an agile or a traditional phased or gated approach. Organizational and team culture can block a smooth transition to an agile approach. In this chapter, we discuss factors that can directly affect a tester’s role on an agile team.

组织文化

Organizational Culture

组织文化由其价值观、规范和假设来定义。一个组织的文化支配着人们如何沟通、相互关联和做出决策,通过观察员工行为很容易看出这一点。

An organizational culture is defined by its values, norms, and assumptions. An organization’s culture governs how people communicate, interrelate, and make decisions, and it is easily seen by observing employee behavior.

组织的文化可以影响敏捷团队的成功。敏捷团队最适合允许独立思考的组织。例如,如果一家公司采用层级结构并鼓励对其所有项目采用指令式管理方式,那么敏捷团队可能会举步维艰。组织过去的经验也会影响新敏捷团队的成功。如果一家公司尝试敏捷但结果不佳,人们会怀疑是否会再次尝试,并举出失败的例子。他们甚至可能会积极反对它。

The culture of an organization can impact the success of an agile team. Agile teams are best suited for organizations that allow independent thinking. For example, if a company has a hierarchical structure and encourages a directive management style for all its projects, agile teams will probably struggle. Past experiences of the organization will also affect the success of a new agile team. If a company tried agile and had poor results, people will be suspicious of trying it again, citing examples of why it didn’t work. They might even actively campaign against it.

在尝试实施敏捷过程时,组织文化经常被忽视,这让人们想知道为什么它没有像承诺的那样工作。很难改变既定的流程,尤其是当个人觉得他们对现状有利害关系时。每个职能部门都发展出满足其需求的亚文化和流程。他们对自己的工作方式感到满意。恐惧是一种强烈的情绪,如果不加以解决,它可能会危及向敏捷的过渡。如果团队成员觉得新的敏捷过程威胁到他们的工作,他们就会抵制这种变化。

Organizational culture is too frequently not considered when attempts are made to implement an agile process, leaving people wondering why it didn’t work as promised. It’s hard to change established processes, especially if individuals feel they have a stake in the status quo. Each functional group develops a subculture and processes that meet their needs. They’re comfortable with the way they work. Fear is a powerful emotion, and if it is not addressed, it can jeopardize the transition to agile. If team members feel that a new agile process threatens their jobs, they’ll resist the change.

我们将具体讨论组织文化如何影响在敏捷环境中工作的测试人员。参考书目包含处理可能影响团队的其他文化方面的资源。

We’ll talk specifically about how organizational culture affects testers working in an agile environment. The bibliography contains resources that deal with other cultural aspects that may affect teams.

品质理念

Quality Philosophy

从组织如何确定可接受的软件质量水平的角度考虑组织的质量理念。它能容忍质量差吗?它是否考虑了客户的质量要求,还是只关心尽快将产品送到客户手中?

Consider an organization’s quality philosophy in terms of how it determines the acceptable level of software quality. Does it tolerate poor quality? Does it take customers’ quality requirements into account, or is it just concerned with getting the product into the customers’ hands as fast as it can?

当一个组织缺乏整体质量理念并迫使团队在不考虑质量的情况下推出产品时,测试人员会感到压力。试图在这样的环境中使用敏捷开发的团队面临着一场艰苦的战斗。

When an organization lacks an overall quality philosophy and pressures teams to get the product out without regard to quality, testers feel the pinch. A team that tries to use agile development in such an environment faces an uphill battle.

一些组织拥有强大的、独立的测试团队,他们拥有很大的权力。这些团队及其经理可能认为敏捷开发会夺走这种力量。他们可能担心敏捷与他们的质量理念背道而驰。评估您组织的质量理念和执行它的团队的理念。

Some organizations have strong, independent test teams that wield a lot of power. These teams, and their managers, might perceive that agile development will take that power away. They might fear that agile runs contrary to their quality philosophy. Evaluate your organization’s quality philosophy and the philosophy of the teams that enforce it.

危险:素质警察心态

Peril: Quality Police Mentality

如果现有的 QA 团队承担了“质量警察”的角色,那么其成员通常会通过确保完成代码审查并将错误认真地输入缺陷跟踪系统来加强质量。他们保留有关发现的错误数量的指标,然后负责做出是否发布产品的最终决定。

If an existing QA team has assumed the role of “Quality Police,” its members usually enforce quality by making sure code reviews are completed and bugs are religiously entered into the defect-tracking systems. They keep metrics about the number of bugs found, and then are charged with making the final decision as to whether to release the product.

我们与测试人员交谈过,他们吹嘘自己的成就,例如越过开发经理的头脑强制程序员遵循编码标准。我们甚至听说过测试人员花时间编写有关不符合他们标准的要求的错误。这种态度不会在协作的敏捷团队中发挥作用。它助长了对抗行为。

We’ve talked to testers who brag about accomplishments such as going over a development manager’s head to force a programmer to follow coding standards. We’ve even heard of testers who spend their time writing bugs about requirements that aren’t up to their standards. This kind of attitude won’t fly on a collaborative agile team. It fosters antagonistic behavior.

“质量警察”角色的另一个风险是团队不接受构建质量的概念,程序员开始使用测试人员作为安全网。团队开始通过错误跟踪系统进行沟通,这不是一种非常有效的沟通方式,因此团队从不“僵持”。

Another risk of the “Quality Police” role is that the team doesn’t buy into the concept of building quality in, and the programmers start using testers as a safety net. The team starts communicating through the bug-tracking system, which isn’t a very effective means of communicating, so the team never “jells.”

继续阅读以了解有助于避免这种危险的方法。

Read on for ways to help avoid this peril.

每个人都重视质量的公司将更容易过渡到敏捷。如果任何一个团队承担了质量责任,他们就必须学会与团队中的其他人分享这一点才能取得成功。

Companies in which everyone values quality will have an easier time transitioning to agile. If any one group has assumed ownership of quality, they’ll have to learn to share that with everyone else on the team in order to succeed.

整个团队的质量所有权

第 1 章中,“到底什么是敏捷测试?”,我们谈到了整个团队的质量方法。对于许多测试人员和 QA 团队来说,这意味着思想从拥有质量转变为在定义和维护质量方面发挥参与作用。对于许多测试人员和 QA 团队来说,如此剧烈的态度转变是困难的。

In Chapter 1, “What Is Agile Testing, Anyway?,” we talked about the whole-team approach to quality. For many testers and QA teams, this means a mind shift from owning quality to having a participatory role in defining and maintaining quality. Such a drastic shift in attitude is difficult for many testers and QA teams.

一直在传统环境中工作的测试人员可能很难适应他们的新角色和活动。如果他们来自开发和 QA 存在敌对关系的组织,则可能很难从事后才想到(如果有的话)转变为团队不可或缺的一部分。程序员和测试人员很难学会相互信任。

Testers who have been working in a traditional setting might have a hard time adjusting to their new roles and activities. If they’ve come from an organization where development and QA have an adversarial relationship, it may be difficult to change from being an afterthought (if thought of at all) to being an integral part of the team. It can be difficult for both programmers and testers to learn to trust each other.

技能和适应能力

关于无法适应敏捷实践的程序员已经有很多观察——但是对于习惯于根据需求文档构建测试脚本的测试人员呢?他们能学会问问题作为代码吗正在建设中?不改变测试方法的测试人员很难与开发团队的其他成员紧密合作。

Much has been observed about programmers who can’t adapt to agile practices—but what about testers who are used to building test scripts according to a requirements document? Can they learn to ask the questions as the code is being built? Testers who don’t change their approach to testing have a hard time working closely with the rest of the development team.

习惯于通过用户界面仅进行手动测试的测试人员可能不了解敏捷开发固有的自动化方法。这些测试人员需要很大的勇气来面对他们不断变化的角色,因为变化意味着在他们的舒适区之外发展新的技能组合。

Testers who are used to doing only manual testing through the user interface might not understand the automated approach that is intrinsic to agile development. These testers need a lot of courage in order to face their changing roles, because changing means developing new skill sets outside their comfort zones.

有帮助的因素

尽管有许多文化问题需要考虑,但大多数 QA 团队都将重点放在流程改进上,敏捷项目通过使用回顾等工具鼓励持续改进和适应性。大多数质量保证专业人员都渴望利用他们所学到的知识并加以改进。这些人具有足够的适应性,不仅可以生存,而且可以在敏捷项目中茁壮成长。

Although there are many cultural issues to consider, most QA teams have a focus on process improvement, and agile projects encourage continuous improvements and adaptability through the use of tools like retrospectives. Most quality assurance professionals are eager to take what they’ve learned and make it better. These people are adaptable enough to not only survive, but to thrive in an agile project.

如果您的组织专注于学习,它将鼓励持续的流程改进。它可能会比那些更重视如何应对危机而不是改进流程的组织更快地采用敏捷。

If your organization focuses on learning, it will encourage continual process improvement. It will likely adopt agile much more quickly than organizations that put more value on how they react to crises than on improving their processes.

如果您是一个没有有效质量理念的组织中的测试人员,您可能很难让质量实践被接受。敏捷方法将为您提供一种机制,用于引入良好的面向质量的实践。

If you are a tester in an organization that has no effective quality philosophy, you probably struggle to get quality practices accepted. The agile approach will provide you with a mechanism for introducing good quality-oriented practices.

测试人员需要时间和培训,就像其他学习敏捷项目的人一样。如果您管理的团队包括测试人员,一定要给他们足够的支持。测试人员通常不会在新项目开始时被引入,然后被期望能够适应已经一起工作了几个月的团队。为了帮助测试人员进行调整,您可能需要引入经验丰富的敏捷测试教练。雇用以前在敏捷团队工作过并且可以担任导师和老师的人将帮助测试人员融入新的敏捷文化,无论他们是与现有团队一起过渡到敏捷,还是加入新的敏捷开发团队。

Testers need time and training, just like everyone else who is learning to work on an agile project. If you’re managing a team that includes testers, be sure to give them plenty of support. Testers are often not brought in at the beginning of a greenfield project and are then expected to just fit into a team that has been working together for months. To help testers adjust, you may need to bring in an experienced agile testing coach. Hiring someone who has previously worked on an agile team and can serve as a mentor and teacher will help testers integrate with the new agile culture, whether they’re transitioning to agile along with an existing team or joining a new agile development team.

可持续步伐

Sustainable Pace

传统的测试团队习惯于在项目结束时进行快速而激烈的测试,这转化为周末和​​晚上的工作。在这个项目结束测试阶段,一些组织定期要求他们的团队每周投入 50、60 或更多小时来努力赶上最后期限。组织通常将加班时间视为衡量个人承诺的标准。这与敏捷价值观相冲突,敏捷价值观围绕着让人们始终把工作做到最好。

Traditional test teams are accustomed to fast and furious testing at the end of a project, which translates into working weekends and evenings. During this end-of-project testing phase, some organizations regularly ask their teams to put in 50, 60, or more hours each week to try to meet a deadline. Organizations often look at overtime as a measure of an individual’s commitment. This conflicts with agile values that revolve around enabling people to do their best work all the time.

在敏捷项目中,我们鼓励您以可持续的速度工作。这意味着团队以一致的速度工作,保持恒定的速度,从而保持高质量标准。新的敏捷团队往往对他们可以完成的事情过于乐观,并且报名参加了太多的工作。经过一两次迭代后,他们学会了注册足够的工作,因此不需要加班来完成他们的任务。每周工作 40 小时是 XP 团队正常的可持续发展速度;如果一周又一周地付出努力,人们就可以在长期内完成最多的工作,同时交付良好的价值。

In agile projects, you are encouraged to work at a sustainable pace. This means that teams work at a consistent pace that sustains a constant velocity that permits maintaining a high-quality standard. New agile teams tend be overly optimistic about what they can accomplish and sign up for too much work. After an iteration or two, they learn to sign up for just enough work so no overtime is needed to complete their tasks. A 40-hour week is the normal sustainable pace for XP teams; it is the amount of effort that, if put in week in and week out, allows people to accomplish the most work over the long haul while delivering good value.

团队可能需要不时地为不可持续的节奏而努力,但这应该是例外,而不是常态。如果需要短期加班,整个团队应该加班。如果是冲刺的最后一天并且有些故事没有测试,那么整个团队都应该熬夜完成测试,而不仅仅是测试人员。使用本书中推荐的实践和技术来学习如何规划测试和开发,并让测试“跟上”编码。在您的团队更好地管理其工作量和速度之前,预算额外的时间以帮助平衡步伐。

Teams might need to work for short bursts of unsustainable pace now and then, but it should be the exception, not the norm. If overtime is required for short periods, the whole team should be working extra hours. If it’s the last day of the sprint and some stories aren’t tested, the whole team should stay late to finish the testing, not just the testers. Use the practices and techniques recommended throughout this book to learn how to plan testing along with development and allow testing to “keep up” with coding. Until your team gets better at managing its workload and velocity, budget in extra time to help even out the pace.

客户关系

Customer Relationships

在传统的软件开发中,开发团队与客户之间的关系更像是供应商关系。即使客户是内部客户,它也更像是两个独立的公司,而不是两个致力于创造商业价值的共同目标的团队。

In traditional software development, the relationship between the development teams and their customers is more like a vendor-supplier relationship. Even if the customer is internal, it can feel more like two separate companies than two teams working on a common goal of producing business value.

敏捷开发依赖于客户或至少是他们的代理人的密切参与。敏捷团队邀请客户进行协作,如果可能的话在同一地点工作,并密切参与开发过程。双方都了解彼此的长处和短处。

Agile development depends on close involvement from customers or, at the very least, their proxies. Agile teams have invited customers to collaborate, work in the same locations if possible, and be intimately involved with the development process. Both sides learn each other’s strengths and weaknesses.

这种关系的变化需要双方都认可,不管客户是内部客户还是外部客户。开放的关系对于敏捷项目的成功至关重要,在敏捷项目中,客户团队和开发团队之间的关系更像是一种伙伴关系,而不是供应商与供应商之间的关系。

This change in the relationships needs to be recognized by both sides, and it doesn’t matter whether the customer is internal or external. An open relationship is critical to the success of an agile project, where the relationship between the customer team and the development team is more like a partnership than a vendor-supplier relationship.

珍妮特的故事

Janet’s Story

在我最近参与的一个大型项目中,客户实际上是五家公司的联合体,其中一家是创建软件的软件公司。每家公司都提供了三名最优秀的领域专家来代表他们的需求。这些现场用户和他们自己的组织之间有定期的沟通,他们也是他们日常工作的团队中不可或缺的一部分。

In a large project I was on recently, the customer was actually a consortium of five companies, with one of them being the software company creating the software. Each of the companies supplied three of their best domain experts to represent their needs. There was regular communication between these on-site users and their own organizations, and they were also an integral part of the team they worked with on a daily basis.

一个由来自所有五家公司的代表组成的指导委员会随时了解进展情况,并在需要在更高级别做出决策时被引入。

A steering committee with representatives from all five companies was kept in the loop on progress and was brought in when decisions needed to be made at a higher level.

——珍妮特

—Janet

拥有一些有代表性的领域专家,同时让所有利益相关者不断了解情况,是开发人员与客户成功协作的一种方法。我们将在第五部分讨论其他内容。客户对于敏捷项目的成功至关重要。他们优先考虑将要构建的内容,并对产品质量拥有最终决定权。测试人员与客户密切合作,了解需求并定义验收测试,以证明满足条件。测试活动是开发团队与客户团队关系的关键。这就是测试专业知识对敏捷团队如此重要的原因。

Having a few representative domain experts, while keeping all stakeholders continually informed, is one approach to successful developer-customer collaboration. We’ll talk about others in Part V. Customers are critical to the success of your agile project. They prioritize what will be built and have the final say in the quality of the product. Testers work closely with customers to learn requirements and define acceptance tests that will prove that conditions of satisfaction are met. Testing activities are key to the development team-customer team relationship. That’s why testing expertise is so essential to agile teams.

组织规模

Organization Size

组织的规模对项目的运行方式和公司结构的成熟度有很大的影响。组织越大,结构越趋于分层。随着自上而下的沟通渠道的发展,报告结构变得具有指导性,并且与技术和业务之间的协作不太兼容。

The size of an organization can have great impact on how projects are run and how the structure of a company matures. The larger the organization, the more hierarchical the structure tends to be. As top-down communication channels are developed, the reporting structures become directive and less compatible with collaboration between technology and business.

沟通挑战

一些敏捷流程提供了促进团队间沟通的方法。例如,Scrum 有“Scrum of Scrums”,来自多个团队的代表每天进行协调。

Some agile processes provide ways to facilitate inter-team communication. For example, Scrum has the “Scrum of Scrums,” where representatives from multiple teams coordinate on a daily basis.

如果您在一个大型组织中工作,其中测试团队或其他专业资源与编程团队是分开的,请努力寻找保持联系的方法。例如,如果您的数据库团队是完全独立的,您需要找到一种与数据库专家密切合作的方式,以便及时获得您需要的东西。

If you work in a large organization where the test teams or other specialized resources are separate from the programming teams, work to find ways to keep in constant touch. For example, if your database team is completely separate, you need to find a way to work closely with the database specialists in order to get what you need in a timely manner.

另一个在大公司中更常见的问题是,客户可能不像在小公司那样容易接近。当您尝试收集需求和示例并寻求让客户参与整个开发周期时,这是一个很大的障碍。一种解决方案是让具有领域专业知识的测试人员或分析师充当客户代理。通信工具也可以帮助处理此类情况。寻找创造性的方法来克服大公司固有的问题。

Another problem that tends to be more common in large companies is that customers might not be as accessible as they are in smaller companies. This is a big obstacle when you try to gather requirements and examples and seek to get customer involvement throughout the development cycle. One solution is to have testers or analysts with domain expertise act as customer proxies. Communication tools can help deal with such situations as well. Look for creative ways to overcome the problems inherent in big companies.

第 16 章,“开始行动”,描述了一个大型组织如何使用功能分析师来缓解由于远程客户引起的问题。

Chapter 16, “Hit the Ground Running,” describes how one large organization uses functional analysts to mitigate problems due to remote customers.

组织内的冲突文化

对于大型软件开发商店,敏捷开发通常首先在一个团队或几个团队中实施。如果您的敏捷团队必须使用其他方法(例如分阶段或门控开发)与其他团队进行协调,那么您将面临一系列额外的挑战。如果一些外部团队往往功能失调,那就更难了。即使整个公司都采用了敏捷,一些团队也比其他团队更成功地完成了转型。

With large software development shops, agile development is often first implemented in one team or just a few teams. If your agile team has to coordinate with other teams using other approaches such as phased or gated development, you have an extra set of challenges. If some of the external teams tend to be dysfunctional, it’s even harder. Even when an entire company adopts agile, some teams make the transition more successfully than others.

您的团队也可能会遇到专家团队的抵制,这些专家团队对他们的特定孤岛感到保护。Lisa 与一个团队交谈,该团队的成员无法从他们公司的配置管理团队获得任何帮助,这显然是一个主要障碍。一些开发团队被禁止直接与客户交谈。

Your team might also run into resistance from specialist teams that are feeling protective of their particular silos. Lisa talked to a team whose members could not get any help from their company’s configuration management team, which was obviously a major obstacle. Some development teams are barred from talking directly to customers.

如果第三方在您的团队正在使用的同一系统上工作,他们的文化也会导致冲突。也许您的团队是第三方,而您正在为客户开发软件。您将需要考虑如何减轻基于文化的差异。第 V 部分更详细地介绍了与其他团队和第三方合作,但这里有一些想法可以帮助您入门。

If third parties are working on the same system your team is working on, their cultures can also cause conflicts. Perhaps your team is the third party, and you’re developing software for a client. You will need to think about how to mitigate culture-based differences. Part V goes into more detail about working with other teams and third parties, but here are a few ideas to get you started.

高级计划如果您必须与其他团队协调,则需要在发布计划期间或迭代开始前花时间与他们合作。您需要时间来调整您自己的流程以与其他人的流程一起工作,并且他们可能需要更改他们的流程以适应您的要求。考虑安排对共享资源(如性能测试专家或负载测试环境)的访问,并围绕其他人的日程安排自己的工作。您的利益相关者可能期望您自己的敏捷过程不包括的某些可交付成果,例如正式的测试计划。一些额外的计划将帮助您克服这些文化差异。

Advanced Planning If you have to coordinate with other teams, you will need to spend time during release planning, or before the start of an iteration, to work with them. You need time to adapt your own processes to work with others’ processes, and they might need to change their processes to accommodate your requests. Consider arranging access to shared resources such as performance test specialists or load test environments, and plan your own work around others’ schedules. Your stakeholders might expect certain deliverables, such as formal test plans, that your own agile process doesn’t include. Some extra planning will help you to work through these cultural differences.

第 15 章,“发布或主题规划中的测试人员活动”和第 16 章,“开始行动”,讨论测试人员可以做什么来帮助规划和与其他团队协调。

Chapter 15, “Tester Activities in Release or Theme Planning,” and Chapter 16, “Hit the Ground Running,” talk about what testers can do to help with planning and coordinating with other teams.

现在行动,稍后道歉我们对提出可能引起麻烦的建议犹豫不决,但通常在大型组织中,官僚主义的轮子如此转动慢慢地,您的团队可能不得不找出并实施自己的解决方案。例如,无法与配置管理团队合作的团队只是实施了自己的内部构建流程,并继续努力将其与官方认可的流程相集成。

Act Now, Apologize Later We hesitate to make suggestions that might cause trouble, but often in a large organization, the bureaucratic wheels turn so slowly that your team might have to figure out and implement its own solutions. For example, the team that couldn’t get cooperation from the configuration management team simply implemented its own internal build process and kept working on getting it integrated with the officially sanctioned process.

如果没有官方渠道可以获得您需要的东西,那么是时候发挥创意了。也许测试人员以前从未直接与客户交谈过。尝试自己安排一次会面,或者找一个可以充当客户代理人或中间人的人。

If there aren’t official channels to get what you need, it’s time to get creative. Maybe testers have never talked directly to customers before. Try to arrange a meeting yourself, or find someone who can act as a customer proxy or go-between.

授权你的团队

Empower Your Team

在敏捷项目中,重要的是每个开发团队都感到有权做出决策。如果您是一名经理,并且希望您的敏捷团队取得成功,请让他们自由地采取行动并做出创造性的反应。组织文化必须适应这种变化,敏捷项目才能成功。

In an agile project, it is important for each development team to feel empowered to make decisions. If you’re a manager and you want your agile teams to succeed, set them free to act and react creatively. The culture of an organization must adapt to this change for an agile project to be successful.

第 4 章,“团队后勤”,更多地讨论了独立的职能团队以及它们如何影响敏捷测试人员。

Chapter 4, “Team Logistics,” talks more about separate functional teams and how they affect the agile tester.

测试/QA 团队成功采用敏捷的障碍

Barriers to Successful Agile Adoption by Test/QA Teams

任何变革都面临成功的障碍。正如我们在上一节中讨论的那样,组织文化可能是需要克服的最大障碍。组织文化一旦确立,就很难改变。它的形成需要时间,一旦到位,员工就会致力于这种文化,这使得它极难改变。

Any change faces barriers to success. Organizational culture, as we discussed in the previous section, might be the largest obstacle to overcome. Once organizational culture has become well established, it’s very hard to change. It took time for it to form, and once in place, employees become committed to the culture, which makes it extremely resistant to alteration.

本节讨论您的测试人员和 QA 团队可能遇到的采用敏捷开发方法的具体障碍。

This section discusses specific barriers to adoption of agile development methods that can be encountered by your testers and QA teams.

失去身份

Loss of Identity

出于多种原因,测试人员坚持独立 QA 团队的概念,但主要原因是恐惧,具体来说:

Testers cling to the concept of an independent QA team for many reasons, but the main reason is fear, specifically:

图像担心他们会失去 QA 身份

Fear that they will lose their QA identity

图像担心如果他们向开发经理报告,他们将失去支持而程序员将获得优先权

Fear that if they report to a development manager, they will lose support and programmers will get priority

图像担心他们缺乏在敏捷团队中工作的技能并会丢掉工作

Fear that they lack the skills to work in an agile team and will lose their jobs

图像担心当他们分散到开发团队时,他们得不到所需的支持

Fear that when they’re dispersed into development teams they won’t get the support they need

图像担心他们和他们的经理会在新组织中迷失方向

Fear that they, and their managers, will get lost in the new organization

我们经常听到 QA 经理提出诸如“我的公司正在实施敏捷开发”之类的问题。我的角色如何适应?” 这与“失去身份”的恐惧直接相关。

We often hear of QA managers asking questions such as, “My company is implementing agile development. How does my role fit in?” This is directly related to the “loss of identity” fears.

第 4 章团队后勤”涵盖了可用于帮助人们适应的想法。

Chapter 4, “Team Logistics,” covers ideas that can be used to help people adapt.

其他角色

Additional Roles

我们从经验中知道,新团队往往缺少可能对其成功至关重要的专家或专业知识。Lisa 的团队遇到了如此大的障碍,唯一能做的就是坐下来问:“我们的团队中缺少什么角色阻碍了我们?我们需要什么?另一个开发人员、另一个测试人员、数据库设计人员?” 我们都知道测试是一个广阔的领域。也许您需要在敏捷团队中有测试经验的人。或者您可能需要一名性能测试专家。花时间分析你的产品需要哪些角色才能成功,这一点很重要,如果你需要从团队外部填补这些角色,那就去做吧。

We know from experience that new teams are often missing specialists or expertise that might be key to their success. Lisa’s team has run into obstacles so large that the only thing to do was sit back and ask, “What role are we missing on our team that is holding us back? What do we need? Another developer, another tester, a database designer?” We all know that testing is a vast field. Maybe you need someone experienced in testing on an agile team. Or maybe you need a performance testing specialist. It’s critical that you take the time to analyze what roles your product needs to be successful, and if you need to fill them from outside the team, do it.

至关重要的是,产品团队中的每个人都了解自己的角色,或者弄清楚自己的角色是什么,因为他们现在是新敏捷团队的一员。这样做需要时间和培训。

It’s critical that everyone already on the product team understand their role or figure out what their role is now that they’re part of a new agile team. Doing this requires time and training.

缺乏培训

Lack of Training

我们在Agile 2007 的“会议中的会议”中主持了一次会议,询问人们在他们的敏捷团队中遇到了哪些与测试相关的问题。一位与会者告诉我们,他们按照敏捷文献所提倡的那样拆分了测试组织。但是,他们在没有任何培训的情况下将测试人员放入开发单元;三个月内,所有测试人员都辞职了,因为他们不了解自己的新角色。通过正确的培训和辅导,可以避免此类问题。

We hosted a session in the “Conference within a Conference” at Agile 2007 that asked people what testing-related problems they were having on their agile teams. One of the attendees told us that they split up their test organization as advocated by the agile literature. However, they put the testers into development units without any training; within three months, all of the testers had quit because they didn’t understand their new roles. Problems like these can be prevented with the right training and coaching.

当我们开始与我们的第一个敏捷团队合作时,没有太多资源可以帮助我们了解敏捷测试人员应该做什么或者我们应该如何与我们的团队一起工作。今天,你可以找到很多实践者,他们可以帮助训练测试人员适应敏捷环境,帮助测试团队进行敏捷转型。本地用户组、会议、研讨会、在线指导和邮件列表都为想要学习的测试人员和管理人员提供了宝贵的资源。在需要时不要害怕寻求帮助。良好的指导会给您带来良好的投资回报。

When we started working with our first agile teams, there weren’t many resources available to help us learn what agile testers should do or how we should work together with our teams. Today, you can find many practitioners who can help train testers to adapt to an agile environment and help test teams make the agile transition. Local user groups, conferences, seminars, online instruction, and mailing lists all provide valuable resources to testers and managers wanting to learn. Don’t be afraid to seek help when you need it. Good coaching gives a good return on your investment.

不理解敏捷概念

Not Understanding Agile Concepts

并非所有的敏捷团队都是一样的。敏捷开发有很多不同的方法,例如 XP、Scrum、Crystal、FDD、DSDM、OpenUP 和各种那些的混合。在我们看来,一些自称为“敏捷”的团队并没有真正实践敏捷。许多团队只是采用对他们有用的做法,而不管原始来源如何,或者他们发明了自己的做法。这很好,但如果他们不遵循任何敏捷核心价值观和原则,我们就会质疑给他们贴上敏捷标签。每月发布和免除文档并不等同于敏捷开发!

Not all agile teams are the same. There are lots of different approaches to agile development, such as XP, Scrum, Crystal, FDD, DSDM, OpenUP, and various mixes of those. Some self-titled “agile” teams are not, in our opinion, really practicing agile. Plenty of teams simply adopt practices that work for them regardless of the original source, or they invent their own. That’s fine, but if they don’t follow any of the core agile values and principles, we question giving them an agile label. Releasing every month and dispensing with documentation does not equate to agile development!

如果不同的团队成员对什么是“敏捷”、他们应该使用哪些实践或应该如何实践这些实践有相反的看法,就会有麻烦。例如,如果你是一名测试人员,正在推动团队实施持续集成,但程序员根本拒绝尝试,那么你就处于不利地位。如果您是一名未能成功参与某些实践的程序员,例如通过面向业务的测试来推动开发,那么您也会陷入冲突。

If different team members have opposing notions of what constitutes “agile,” which practices they should use, or how those practices are supposed to be practiced, there’s going to be trouble. For example, if you’re a tester who is pushing for the team to implement continuous integration, but the programmers simply refuse to try, you’re in a bad spot. If you’re a programmer who is unsuccessful at getting involved in some practices, such as driving development with business-facing tests, you’re also in for conflict.

团队必须就如何继续进行达成共识,以便成功过渡到敏捷。许多敏捷开发实践是协同的,因此如果单独使用它们,它们可能无法提供团队正在寻找的好处。也许团队可以同意对给定迭代次数的某些实践进行试验并评估结果。它可以决定寻求外部输入,以帮助他们了解这些做法以及它们如何组合在一起。不同的观点对团队有好处,但每个人都需要朝着同一个方向前进。

The team must reach consensus on how to proceed in order to make a successful transition to agile. Many of the agile development practices are synergistic, so if they are used in isolation, they might not provide the benefits that teams are looking for. Perhaps the team can agree to experiment with certain practices for a given number of iterations and evaluate the results. It could decide to seek external input to help them understand the practices and how they fit together. Diverse viewpoints are good for a team, but everyone needs to be headed in the same direction.

我们采访过的一些人描述了传统软件开发组织实施敏捷开发过程时经常出现的“迷你瀑布”现象。该组织用两周或四个星期的开发周期取代了六个月或一年的开发周期,并试图将所有传统的 SDLC 阶段压缩到这么短的时间内。自然地,他们继续遇到与以前相同的问题。图 3-1显示了迷你瀑布的“理想”版本,其中有一个代码和修复阶段,然后是测试——测试在编码完成之后但在下一次迭代开始之前进行。然而,真正发生的是测试被挤到迭代的末尾并且通常拖到下一个迭代中。程序员还没有太多需要解决的问题,所以他们开始着手下一次迭代。不久之后,一些团队的测试总是“落后”一个迭代,并且发布日期像往常一样被推迟。

Several people we’ve talked to described the “mini-waterfall” phenomenon that often occurs when a traditional software development organization implements an agile development process. The organization replaces a six-month or year-long development cycle with a two- or four-week one, and just tries to squeeze all of the traditional SDLC phases into that short period. Naturally, they keep having the same problems as they had before. Figure 3-1 shows an “ideal” version of the mini-waterfall where there is a code-and-fix phase and then testing—the testing comes after coding is completed but before the next iteration starts. However, what really happens is that testing gets squeezed into the end of the iteration and usually drags over into the next iteration. The programmers don’t have much to fix yet, so they start working on the next iteration. Before long, some teams are always an iteration “behind” with their testing, and release dates get postponed just as they always did.

图 3-1一个迷你瀑布流程

Figure 3-1 A mini-waterfall process

图像

参与交付产品的每个人都需要时间和培训来理解敏捷背后的概念以及核心实践。经验丰富的教练可用于对团队的新实践进行实践培训,例如测试驱动开发。在较大的组织中,职能测试经理可以成为实践主管,并可以提供支持和资源,以便测试人员学习如何与他们的新团队进行沟通和协作。程序员和其他团队成员需要来自职能经理的类似帮助。强大的领导力将帮助团队找到从“迷你瀑布”迁移到真正的协作开发的方法,其中编码和测试集成到一个过程中。

Everyone involved with delivering the product needs time and training to understand the concepts behind agile as well as the core practices. Experienced coaches can be used to give hands-on training in practices new to the team, such as test-driven development. In larger organizations, functional test managers can become practice leads and can provide support and resources so that testers learn how to communicate and collaborate with their new teams. Programmers and other team members need similar help from their functional managers. Strong leadership will help teams find ways to migrate away from “mini-waterfall” to true collaborative development, where coding and testing are integrated into one process.

请参阅参考书目以获取有关 XP 雷达图的更多信息的链接。

See the bibliography for a link to more information about XP Radar charts.

XP 开发了一个雷达图来帮助团队确定他们对关键 XP 实践的适应程度。他们测量了五种不同的关键实践:团队、编程、规划、客户和结对,并显示了团队对实践的适应程度。图 3-2显示了两个这样的图表。图表左图显示适配成功,而右图显示存在一些问题区域。

XP has developed a radar chart to help teams determine their level of adaptation to key XP practices. They measure five different key practices: team, programming, planning, customer, and pairing, and they show the level of adaptation to practices by teams. Figure 3-2 shows two such charts. The chart on the left shows successful adaptation, while the chart on the right shows that there are some problem areas.

图 3-2 XP 雷达图

Figure 3-2 XP Radar charts

图像

过去的经验/态度

Past Experience/Attitude

很多人都经历过没有坚持下去的改变。一些开发组织经历了一系列“流行方法论”。他们举起手来想:“我们为什么要再做一次?” 人们陷入了旧的、不成功的模式。即使当他们尝试新事物时,他们也可能在压力下恢复旧习惯。以下只是一些人因过去的经验和他们对“事物的现状”的看法而拒绝改变的几个例子:

Lots of people have been through changes that didn’t stick. Some development organizations have lived through a succession of the “methodology du jour.” They throw up their hands and wonder, “Why should we do it again?” People get stuck in their old, unsuccessful patterns. Even when they try something new, they might revert to bad old habits when under stress. The following are just a few examples of people resisting change due to past experience and their perception of “the way things are”:

图像测试人员坐在他的立方体中,不会与程序员谈论他遇到的问题。他抱怨程序员不明白他想要什么。

A tester sat in his cube and wouldn’t talk with the programmers about problems he was having. He complained that programmers didn’t understand what he wanted.

图像测试人员无法摆脱他现有的态度,即程序员不知道如何编写好的代码,或者如何测试它。他居高临下的态度有目共睹,他作为测试人员的信誉受到了挑战。

A tester couldn’t shake his existing attitude that programmers didn’t know how to write good code, or how to test it. His condescending attitude was clear to all, and his credibility as a tester was challenged.

图像当程序员做了他不喜欢的事情时,一位客户举起了手,因为他们“总是”做他们想做的事。

A customer threw up his hands when the programmers did something he didn’t like, because they “always” do what they want anyhow.

当面临向敏捷开发的过渡时,像这样的人往往不给新流程机会就离开。敏捷开发并不适合所有人,但培训和试验时间可以帮助调整态度。要求每个人都成为解决方案的一部分,并共同努力找出最适合他们特定情况的流程和实践。自组织团队可以成为一个强大的工具,可以用来让开发团队的所有成员确信他们可以控制自己的命运。

When faced with a transition to agile development, people like this often leave without giving the new process a chance. Agile development isn’t for everyone, but training and time to experiment can help adjust attitudes. Ask everyone to be part of the solution, and work together to find out what processes and practices work best for their particular situations. The self-organizing team can be a powerful tool to use to reassure all members of the development team that they’re in control of their own destiny.

角色之间的文化差异

Cultural Differences among Roles

每个新的敏捷团队成员都在从不同的角度进行转变。程序员通常习惯于编写产品代码并尽快发布。系统管理员和数据库专家可能习惯于在自己的孤岛中工作,按照自己的时间表执行请求。客户可能从未直接与开发团队成员交谈过。测试人员可能习惯于在项目结束时进来,根本不与程序员进行太多交互。

Each new agile team member is making the transition from a different perspective. Programmers are often used to writing production code and getting it released as quickly as possible. System administrators and database experts might be accustomed to working in their own silo, performing requests on their own schedule. Customers may never have talked directly with development team members. Testers might be used to coming in at the end of the project and not interacting much at all with programmers.

难怪向敏捷过渡会令人恐惧。团队可以制定规则和指南来帮助他们更好地沟通和合作。为了例如,Lisa 加入了一个新的敏捷团队,该团队的规则是如果有人要你与她结对,你必须同意。你可能无法在那一刻做对,但一旦你能解放自己,你就必须去帮助你的队友。

It’s no wonder a transition to agile can be scary. Teams can come up with rules and guidelines to help them communicate and work well together. For example, Lisa joined a new agile team whose rule was that if someone asked you to pair with her, you had to agree. You might not be able to do it right that minute, but as soon as you could free yourself up, you had to go help your teammate.

确定从事不同活动的人们需要什么,并找到提供这些需求的方法。客户需要某种方式来了解开发的进展情况以及是否满足他们的满意条件。开发人员需要了解业务优先级和要求。测试人员需要获取示例并将其转化为测试的方法。所有团队成员都希望感到自己是有价值的、一流的团队成员。每个团队成员还需要有安全感,可以自由地提出问题和尝试新想法。了解每个角色的观点有助于团队完成过渡。

Identify what people doing different activities need, and find ways to provide it. Customers need some way to know how development is progressing and whether their conditions of satisfaction are being met. Developers need to know business priorities and requirements. Testers need ways to capture examples and turn them into tests. All team members want to feel they are valued, first-class team members. Each team member also needs to feel safe and to feel free to raise issues and try new ideas. Understanding the viewpoint of each role helps teams through the transition.

引入变革

Introducing Change

实施任何更改时,请注意副作用。第一阶段可能是混乱;您的团队不确定新流程是什么,有些团队忠于旧方法,有些人不确定且具有破坏性。人们将这个混乱的阶段误认为是新的现状。为避免这种情况,请预先解释变更模型并设定期望。在实施敏捷流程时,期待并接受感知到的混乱。找到最痛苦的领域,并确定哪些做法可以解决问题,这样您就可以从混乱中取得一些立竿见影的进展。

When implementing any change, be aware of the side effects. The first stage may be chaos; your team isn’t sure what the new processes are, some groups are loyal to old ways, and some people are unsure and disruptive. People mistake this chaotic stage for the new status quo. To avoid this, explain the change model up front and set expectations. Expect and accept perceived chaos as you implement agile processes. Find the areas of the most pain, and determine what practices will solve the problem so that you can get some immediate progress out of the chaos.

谈论恐惧

Talk about Fears

当您开始迭代开发时,使用回顾为人们提供一个谈论他们恐惧的地方以及一个他们可以提供反馈的地方。让人们知道害怕是正常的。开放;教他们说他们害怕或不舒服是可以接受的。讨论每个恐惧的来源,从讨论中学习,做出决定,然后继续前进。恐惧是对变化的普遍反应。强迫人们做他们不想做的事情不利于积极的改变。以身作则。

When you start iterative development, use retrospectives to provide people with a place to talk about their fears and a place in which they can give feedback. Let people know that it’s normal to be fearful. Be open; teach them it is acceptable to say they are fearful or uncomfortable. Discuss each source of fear, learn from the discussion, make decisions, and move on. Fear is a common response to change. Forcing people to do something they don’t want is detrimental to positive change. Lead by example.

丽莎的故事

Lisa’s Story

Janet 和我各自加入了我们的第一个 XP 团队,当时许多 XP 从业者认为 XP 团队中没有测试人员的位置。XP 有“客户权利法案”和“程序员权利法案”,但明显没有“测试人员权利法案”。Tip House 和我提出了我们自己的“测试人员权利法案”,以便为测试人员提供在敏捷团队中取得成功的支持和勇气。多年来,许多测试人员告诉我们这对他们和他们的团队了解测试人员如何与其他团队成员一起工作有多大帮助。我不喜欢太多的规则,但他们可以当他们帮助团队克服文化障碍并了解如何以新的方式工作时,这是一件好事。以下列表显示了“测试者权利法案”。我们鼓励您使用它来帮助测试人员融入敏捷团队。

Janet and I each joined our first XP teams at a time when many XP practitioners didn’t see any place for testers on an XP team. XP had a “Customer Bill of Rights” and a “Programmer Bill of Rights,” but the “Tester Bill of Rights” was conspicuously absent. Tip House and I came up with our own “Tester Bill of Rights” in order to give testers the support and courage to succeed on agile teams. Over the years, many testers have told us how much this helped them and their teams learn how testers work together with other team members. I don’t like too many rules, but they can be a good thing when they help the team to overcome cultural barriers and to understand how to work in new ways. The following list presents a “Tester Bill of Rights.” We encourage you to use it to help testers integrate into agile teams.

• 您有权随时提出与测试、质量和流程相关的问题。

• You have the right to bring up issues related to testing, quality, and process at any time.

• 您有权向客户、程序员和其他团队成员提出问题并得到及时答复。

• You have the right to ask questions of customers, programmers, and other team members and receive timely answers.

• 您有权向项目团队中的任何人(包括程序员、经理和客户)寻求和接受帮助。

• You have the right to ask for and receive help from anyone on the project teams, including programmers, managers, and customers.

• 您有权估算测试任务并将其包含在故事估算中。

• You have the right to estimate testing tasks and have these included in story estimates.

• 您有权获得及时执行测试任务所需的工具。

• You have the right to the tools you need to perform testing tasks in a timely manner.

• 您有权期望您的整个团队,而不仅仅是您自己,对质量和测试负责。

• You have the right to expect your entire team, not just yourself, to be responsible for quality and testing.

——丽莎

—Lisa

赋予团队所有权

Give Team Ownership

一个关键的成功因素是团队是否拥有所有权并有能力定制其方法。如果给予正确的帮助,人们可以改变他们的态度和看法。丽莎能够观察迈克·科恩 (Mike Cohn) 作为教练与她的团队一起工作。作为一个自组织团队,该团队必须找出并解决自己的问题。迈克确保他们有时间和资源来试验和改进。他确保企业了解质量比数量或速度更重要。每个团队,即使是自组织团队,都需要一位能够与组织的管理团队有效互动的领导者。

A critical success factor is whether the team takes ownership and has the ability to customize its approach. People can change their attitudes and their perceptions if they are given the right help. Lisa was able to observe Mike Cohn work with her team as a coach. As a self-organizing team, the team had to identify and solve its own problems. Mike made sure they had the time and resources to experiment and improve. He made sure that the business understood that quality was more important than quantity or speed. Every team, even a self-organizing team, needs a leader who can effectively interact with the organization’s management team.

庆祝成功

Celebrate Success

实施变革需要时间并且可能令人沮丧,因此请务必庆祝您的团队取得的所有成功。当您在迭代的第四天达到为所有故事编写高级测试用例的目标时,请拍拍自己的背。当您刚刚交付了一个迭代的工作量时,召集团队一起玩问答游戏或共进午餐。如果你想让改变坚持下去,承认是很重要的。

Implementing change takes time and can be frustrating, so be sure to celebrate all successes your team achieves. Pat yourselves on the back when you meet your goal to write high-level test cases for all stories by the fourth day of the iteration. Get the team together for a trivia game or lunch when you’ve just delivered an iteration’s worth of work. Acknowledgment is important if you want a change to stick.

第 18 章,“编码和测试”,介绍了测试人员和程序员如何在整个开发过程中协同工作。

Chapter 18, “Coding and Testing,” covers how testers and programmers work together throughout the development process.

将测试人员集成到开发团队中,同时让他们继续向提供支持的 QA 经理报告是简化向敏捷开发过渡的一种方法。测试人员可以找到从与程序员的敌对关系转变为协作关系的方法。他们可以展示他们如何提供帮助团队了解客户的需求并提供适当的商业价值。他们可以举办愉快的活动来建立良好的团队互动。为队友准备饼干或巧克力是让他们走到你办公桌前的好方法!耐心和乐趣是很大的优势。

Integrating testers into development teams while letting them continue to report to a supportive QA manager is one way to ease the transition to agile development. Testers can find ways to move from an adversarial relationship with programmers to a collaborative one. They can show how they can help the team understand the customers’ needs and deliver appropriate business value. They can host enjoyable activities to build good team interactions. Having cookies or chocolate available for teammates is a good way to get them to walk over to your desk! Patience and a sense of fun are big advantages.

管理层期望

Management Expectations

当我们想到采用敏捷所涉及的挑战时,我们通常会想到实际的团队及其遇到的问题。然而,要成功采用敏捷,管理层的支持至关重要。在分阶段项目中,管理层会定期获得更新和签署文件,指示每个阶段的结束。上层管理人员可能不了解他们将如何衡量敏捷项目的进度。他们可能害怕失去控制或缺乏“过程”。

When we think of challenges involved with adopting agile, we generally think of the actual team and the issues it encounters. However, for successful agile adoption, management buy-in is critical. In a phased project, management gets regular updates and sign-off documents indicating the end of each phase. Upper-level managers might not understand how they’ll be able to gauge agile project progress. They might fear a loss of control or lack of “process.”

管理者的文化变革

Cultural Changes for Managers

在敏捷项目中,期望会发生变化。在她以前的瀑布项目中,Janet 记得几周以来一直听到诸如“此功能已完成 90%”之类的评论。这些类型的指标在敏捷项目中毫无意义。没有签字来标记一个阶段的结束,并且项目的“完成”不是由门来衡量的。

In an agile project, expectations change. In her previous life in waterfall projects, Janet remembers hearing comments like “this feature is 90% done” for weeks. Those types of metrics are meaningless in agile projects. There are no sign-offs to mark the end of a phase, and the “doneness” of a project isn’t measured by gates.

有意义的指标由每个项目团队确定。在 Scrum 中,冲刺和发布燃尽图跟踪故事的完成情况,并可以为经理提供进度衡量标准,但不是用于向客户开账单的任何硬性“日期”。测试矩阵可用于跟踪功能测试覆盖率,但不提供签核文档。

Meaningful metrics are determined by each project team. In Scrum, sprint and release burndown charts track story completion and can give managers a measure of progress, but not any hard “dates” to use for billing customers. Test matrices can be used to track functionality test coverage but do not provide sign-off documentation.

一些经理难以理解的另一个变化是让团队做出自己的技术决策并管理自己的工作量。不再是经理决定什么是足够好的。团队(包括客户)定义了交付成功应用程序所需的质量水平。

The other change that is difficult for some managers to understand is letting the teams make their own technical decisions and manage their own workloads. It’s no longer the manager who decides what is good enough. It is the team (which includes the customer) that defines the level of quality necessary to deliver a successful application.

敏捷团队比传统团队估计和工作的时间更短。团队需要为良好的设计和执行计划足够的时间,而不是建立应急计划,以确保技术债务不会增加。敏捷团队的管理者不是在低层级管理团队的活动,而是专注于消除障碍,以便团队成员能够做到最好。

Agile teams estimate and work in smaller chunks of time than traditional teams. Rather than building in contingency, teams need to plan enough time for good design and execution in order to ensure that technical debt does not increase. Rather than managing the team’s activities at a low level, managers of agile teams focus on removing obstacles so that team members can do their best work.

珍妮特的故事

Janet’s Story

我问负责一个大型敏捷项目的副总裁,从管理的角度来看,他发现在新的敏捷环境中最困难的部分是什么。他说,在传统的瀑布式项目中,报告都显示一切都按计划进行,直到最后,然后一切都处于恐慌状态,“什么都没做”。

I asked the vice president in charge of a large agile project what he found to be the most difficult part in the new agile environment from a management perspective. He said that in a traditional waterfall type project, the reports all showed that everything was going according to plan until the very end, and then everything was in a panic state and “nothing worked.”

在敏捷项目中,每天都有问题需要解决。敏捷项目更多的是在一致的基础上工作,但至少他得到了切合实际的报告。项目结束时没有任何意外。

In the agile project, there were problems every day that needed to be addressed. Agile projects were more work on a consistent basis, but at least he was getting realistic reports. There were no surprises at the end of the project.

——珍妮特

—Janet

业务利益相关者不喜欢惊喜。如果可以说服他们给团队足够的时间和资源来进行过渡,他们会发现敏捷开发可以让他们更准确地规划并以稳定的增量实现业务目标。

Business stakeholders don’t like surprises. If they can be convinced to give the team enough time and resources to make the transition, they’ll find that agile development lets them plan more accurately and achieve business goals in steady increments.

有时实际上是管理层决定开始进行敏捷开发。Lisa 公司的业务负责人选择尝试敏捷开发,以解决其软件危机。为了有效,他们需要有一套不同的管理期望。他们需要对做出重大改变的困难保持敏感,尤其是在一个运作不佳的组织中。

Sometimes it’s actually management that drives the decision to start doing agile development. The business leaders at Lisa’s company chose to try agile development in order to solve its software crisis. To be effective, they needed to have a different set of management expectations. They needed to be sensitive to the difficulty of making big changes, especially in an organization that wasn’t functioning well.

在所有情况下,在向高效敏捷团队过渡的漫长过程中,管理者都需要很大的耐心。他们的工作是确保他们提供必要的资源,并确保每个人都能学习如何完成高质量的工作。

In all cases, managers need lots of patience during what might be a long transition to a high-functioning agile team. It’s their job to make sure they provide the necessary resources and that they enable every individual to learn how to do high-quality work.

如果您是 QA 经理,请准备好帮助您的测试人员克服从定义的、顺序的测试阶段转移到他们在任何一天执行广泛不同任务的快节奏迭代时的挫败感。帮助他们适应这样的想法,即测试不再是开发之后发生的单独活动,而是测试和编码是集成的活动。

If you’re a QA manager, be prepared to help your testers overcome their frustrations with moving from defined, sequential testing stages to fast-paced iterations where they perform widely varied tasks on any given day. Help them adapt to the idea that testing is no longer a separate activity that occurs after development but that testing and coding are integrated activities.

如果您是一名测试人员或其他团队成员,在向敏捷开发过渡的过程中没有获得所需的支持,请考虑您的经理在理解敏捷开发方面可能遇到的困难。帮助他们了解您需要什么样的支持。

If you’re a tester or other team member who isn’t getting the support you need in your transition to agile development, think about the difficulties your managers might be having in understanding agile development. Help them to understand what kinds of support you need.

说经理的语言

Speaking the Manager’s Language

企业管理者最了解什么?这是底线——ROI (投资回报率)。要从您的管理层那里获得您需要的支持,请在他们可以理解的背景下构建您的需求。您团队的速度转化为新功能,使业务更有利可图。如果您需要时间和资金来学习和实施自动化测试工具,请向管理层解释随着时间的推移,自动化回归测试将使您的团队更快地进行并在每次迭代中提供更多功能。

What do business managers understand best? It’s the bottom line—the ROI (return on investment). To get the support you need from your management, frame your needs in a context that they can understand. Your team’s velocity translates into new features to make the business more profitable. If you need time and funds to learn and implement an automated test tool, explain to management that over time, automated regression tests will let your team go faster and deliver more functionality in each iteration.

丽莎的故事

Lisa’s Story

我的团队需要大量时间来进行有风险的重构,例如尝试将代码库拆分为多个可以独立构建的模块。我们还需要时间将现有工具升级到最新版本,或者试用新工具。当我们还试图为业务交付故事时,所有这些任务都很难集成到为期两周的冲刺中。

My team needs big blocks of time to do risky refactoring, such as trying to split the code base into multiple modules that can be built independently. We also need time to upgrade to the latest versions of our existing tools, or to try out new tools. All of these tasks are difficult to integrate into a two-week sprint when we’re also trying to deliver stories for the business.

我们向我们的管理层解释说,如果这些“工程”任务被推迟太久,我们的技术债务就会累积,我们的速度就会减慢。每次迭代交付的故事点数会下降,新故事的编码时间会更长。企业需要越来越长的时间才能获得吸引客户所需的新功能。

We explained to our management that if these “engineering” tasks were put off too long, our technical debt would accumulate and our velocity would slow. The number of story points delivered each iteration would decline, and new stories would take longer to code. It would take longer and longer for the business to get the new features it needed in order to attract customers.

业务部门很难同意让我们每六个月进行一次为期两周的迭代来完成我们管理技术债务所需的内部工作,但随着时间的推移,他们可以看到我们速度的结果。最近,一位经理实际上问我们是否需要更频繁地进行“工程冲刺”。产品和团队都在成长,企业希望确保我们也发展我们的基础设施和工具。

It was hard for the business to agree to let us devote a two-week iteration every six months to do the internal work we needed to manage our technical debt, but over time they could see the results in our velocity. Recently, one of the managers actually asked if we might need to have “engineering sprints” more often. Both the product and the team are growing, and the business wants to make sure we grow our infrastructure and tools, too.

——丽莎

—Lisa

与敏捷团队的所有成员一样,管理人员需要学习很多新概念并弄清楚他们如何适应团队成员。使用大的可见图表(或根据需要使用它们的虚拟等价物)以确保它们可以跟踪每次迭代和发布的进度。寻找最大化投资回报率的方法。通常,当存在可提供类似价值的更简单、更快速的解决方案时,企业会要求提供复杂且昂贵的功能。确保你解释了你的团队的工作如何影响底线。与他们合作,为利益相关者找到表达每个新功能需求的最佳方式。

Like all members of an agile team, managers need to learn a lot of new concepts and figure out how they fit as team members. Use big visible charts (or their virtual equivalents, as needed) to make sure they can follow the progress of each iteration and release. Look for ways to maximize ROI. Often, the business will ask for a complex and expensive feature when there is a simpler and quicker solution that delivers similar value. Make sure you explain how your team’s work affects the bottom line. Collaborate with them to find the best way for stakeholders to express the requirements for each new feature.

预算限制是大多数团队面临的现实。当资源有限时,您的团队需要更具创造力。整个团队的方法会有所帮助。也许,像 Lisa 的团队一样,您的团队购买软件的预算有限,所以您倾向于查看通常没有大量前期购买成本的开源测试自动化工具。与应用程序使用相同语言的工具不会帮助非编程测试人员,除非程序员与他们协作以自动化测试。利用团队的所有专业知识帮助您在业务限制范围内工作。

Budget limitations are a reality most teams face. When resources are limited, your team needs to be more creative. The whole-team approach helps. Perhaps, like Lisa’s team, your team has a limited budget to buy software, and so you tend to look at open-source test automation tools that usually don’t have a large up-front purchase cost. A tool that uses the same language as the application won’t help the non-programming testers unless the programmers collaborate with them to automate the tests. Leveraging all of the expertise on the team helps you work within the business limitations.

与您的团队遇到的所有挑战一样,尝试开发团队和管理层可以互相帮助以构建有价值的产品的新方法。同时,无论您采用何种开发方法,您可能都必须确保某些过程(如符合审计要求)得到必要的关注。

As with all challenges your team encounters, experiment with new ways that the development team and management can help each other to build a valuable product. At the same time, regardless of your development approach, you might have to make sure that some processes, such as conformance to audit requirements, receive the necessary attention.

改变来之不易

Change Doesn’t Come Easy

敏捷开发可能看起来节奏很快,但变化似乎很缓慢。刚接触敏捷的团队在掌握他们承诺使用的一些实践方面会很慢。我们遇到过许多测试人员,他们对他们的“敏捷”开发周期实际上是迷你瀑布周期感到沮丧。这些测试人员仍然受到挤压;它只是更频繁地发生。在可以测试故事之前,迭代就结束了。程序员拒绝或无法采用 TDD 或结对等关键实践。团队将质量责任交给测试人员,他们无力更改流程。

Agile development might seem fast-paced, but change can seem glacial. Teams that are new to agile will be slow to master some practices they’ve committed to using. We’ve met many testers who are frustrated that their “agile” development cycles are actually mini-waterfall cycles. These testers are still getting squeezed; it just happens more often. Iterations are over before stories can be tested. Programmers refuse or aren’t able to adopt critical practices such as TDD or pairing. The team leaves responsibility for quality in the hands of the testers, who are powerless to make changes to the process.

没有任何魔法可以让您的团队做出积极的改变,但我们为希望让他们的团队以积极的方式改变的测试人员提供了一些技巧。

There’s no magic that you can use to get your team to make positive changes, but we have some tips for testers who want to get their teams to change in positive ways.

要有耐心

Be Patient

TDD 等新技能很难。想方设法帮助您的团队腾出时间来掌握它们。在等待期间找到您可以独立进行的更改。例如,当程序员学习编写单元测试时,实现一个 GUI 测试工具,您只需很少的帮助即可使用。帮助团队迈出第一步。请记住,当人们恐慌时,他们会回到旧习惯,即使这些习惯不起作用。专注于微小的积极增量。

New skills such as TDD are hard. Find ways to help your team get time to master them. Find changes you can make independently while you wait. For example, while programmers learn to write unit tests, implement a GUI test tool that you can use with minimal help. Help the team make baby steps. Remember that when people panic, they go back to their old habits, even though those habits didn’t work. Focus on tiny positive increments.

让他们感到痛苦

Let Them Feel Pain

有时你只需要看火车残骸。如果您的改进建议遭到拒绝,并且团队失败了,请再次提出您的建议并要求团队考虑尝试几次迭代。人们最愿意在他们感到最痛苦的地方做出改变。

Sometimes you just have to watch the train wreck. If your suggestions for improvement were rebuffed, and the team fails, bring your suggestion up again and ask the team to consider trying it for a few iterations. People are most willing to change in the areas where they feel the most pain.

建立你的信誉

Build Your Credibility

您现在可能正在与以前没有与测试人员密切合作的程序员一起工作。向他们展示您可以提供哪些帮助。带着你发现的问题去找他们,而不是打开错误报告。请他们在签入之前与您一起检查代码。当他们意识到您贡献了真正的价值时,他们就更有可能倾听您的想法。

You might now be working with programmers who haven’t worked closely with testers before. Show them how you can help. Go to them with issues you’ve found rather than opening bug reports. Ask them to review code with you before they check it in. When they realize you’re contributing real value, they’re more likely to listen to your ideas.

致力于你自己的职业发展

Work On Your Own Professional Development

阅读书籍和文章,参加用户组会议,学习新工具或脚本语言。开始学习您的应用程序的编码语言,并询问您团队中的程序员是否可以与他们结对或者他们是否会辅导您。您的同事会尊重您提高技能的愿望。如果您的本地用户组愿意听您关于敏捷测试的演讲,或者软件通讯发表了您的自动化文章,您的队友可能会注意到您也有一些值得一听的东西。

Read books and articles, go to user group meetings and conferences, and learn a new tool or scripting language. Start learning the language your application is coded in, and ask the programmers on your team if you can pair with them or if they’ll tutor you. Your coworkers will respect your desire to improve your skills. If your local user group is willing to listen to your presentation on agile testing, or a software newsletter publishes your automation article, your teammates might notice you have something worth hearing too.

当心素质警察的心态

Beware the Quality Police Mentality

成为合作者,而不是执行者。如果程序员不遵循编码标准,您可能会感到烦恼,但确保他们这样做不是您的工作。向团队提出您的问题并寻求他们的帮助。如果他们忽视了一个真正伤害团队的关键问题,你可能需要向你的教练或经理寻求帮助。但是要以“请帮我找到解决方案”的方式来做,而不是“让这些人表现得好”。如果您看到一个问题,其他人也很可能看到它。

Be a collaborator, not an enforcer. It might bug you if programmers don’t follow coding standards, but it’s not your job to make sure that they do so. Raise your issues with the team and ask for their help. If they ignore a critical problem that is really hurting the team, you might need to go to your coach or manager for help. But do that in a “please help me find a solution” vein rather than a “make these people behave” one. If you’re seeing a problem, chances are high that others see it too.

用脚投票

Vote with Your Feet

你一直很有耐心。你能想到的方法你都试过了,但是你的管理层不懂敏捷开发。程序员仍然会“翻墙”抛出有缺陷的、无法测试的代码,尽管你尽了最大努力,包括每天工作 14 小时,但这些代码仍按原样发布。没有人关心质量,尽管你尽了最大努力,你还是觉得自己被忽视了。也许是时候寻找一个更好的团队了。有些团队对他们现在的样子很满意,只是因为没有足够的痛苦而不想改变。丽莎所在的团队在混乱中茁壮成长,因为经常有机会找出服务器崩溃的原因并成为英雄。尽管使用敏捷实践成功完成了一个项目,但他们又回到了老习惯,Lisa 最终放弃了改变他们的尝试。

You’ve been patient. You’ve tried every approach you can think of, but your management doesn’t understand agile development. The programmers still throw buggy, untestable code “over the wall,” and that code is released as is despite your best efforts, including working 14-hour days. Nobody cares about quality, and you feel invisible despite your best efforts. It might be time to look for a better team. Some teams are happy the way they are and simply don’t feel enough pain to want to change. Lisa worked on a team that thrived on chaos, because there were frequent opportunities to figure out why the server crashed and be a hero. Despite a successful project using agile practices, they went back to their old habits, and Lisa finally gave up trying to change them.

请参阅参考书目,了解有关成为团队有效变革推动者的一些好资源。

See the bibliography for some good resources on being an effective change agent for your team.

概括

Summary

在本章中,我们讨论了文化问题如何影响测试人员及其团队能否成功过渡到敏捷开发。

In this chapter, we talked about how cultural issues can affect whether testers and their teams can make a successful transition to doing agile development.

图像在做出任何改变之前考虑组织文化。

Consider organizational culture before making any kind of change.

图像当整个组织都重视质量时,测试人员更容易融入敏捷团队,但具有“质量警察”心态的测试人员会遇到困难。

Testers have an easier time integrating into agile teams when their whole organization values quality, but testers with a “quality police” mind-set will struggle.

图像一些测试人员可能难以适应“整个团队”的质量所有权,但团队方法有助于克服文化差异。

Some testers might have trouble adjusting to the “whole team” ownership of quality, but a team approach helps overcome cultural differences.

图像客户团队和开发团队必须紧密合作,我们展示了测试人员如何成为促进这种关系的关键。

Customer teams and developer teams must work closely together, and we showed how testers can be key in facilitating this relationship.

图像往往拥有更多孤立专家团队的大型组织在沟通和协作等领域面临着特殊的文化挑战。

Large organizations that tend to have more isolated specialist teams face particular cultural challenges in areas such as communication and collaboration.

图像测试人员成功采用敏捷的主要障碍包括恐惧、失去身份、缺乏培训、以前对新开发过程的负面体验以及角色之间的文化差异。

Major barriers to success for testers for agile adoption include fear, loss of identity, lack of training, previous negative experiences with new development processes, and cultural differences among roles.

图像为了帮助引入变革和促进沟通,我们建议鼓励团队成员讨论恐惧并庆祝每一次成功,无论多么小。

To help introduce change and promote communication, we suggest encouraging team members to discuss fears and celebrating every success, no matter how small.

图像诸如“测试人员权利法案”之类的指南让测试人员有信心提出问题,并帮助他们在学习和尝试新想法时感到安全。

Guidelines such as a “Tester Bill of Rights” give testers confidence to raise issues and help them feel safe as they learn and try new ideas.

图像经理们面临着他们自己的文化挑战,他们需要提供支持和培训来帮助测试人员在敏捷团队中取得成功。

Managers face their own cultural challenges, and they need to provide support and training to help testers succeed on agile teams.

图像测试人员可以通过提供经理跟踪进度和确定 ROI 所需的信息来帮助团队满足经理的期望。

Testers can help teams accommodate manager expectations by providing the information managers need to track progress and determine ROI.

图像改变来之不易,所以要有耐心,并努力提高自己的技能,这样才能帮助您的团队。

Change doesn’t come easy, so be patient, and work on improving your own skills so you can help your team.

第四章团队后勤

Chapter 4 Team Logistics

图像

敏捷团队强调面对面的沟通对于项目的成功至关重要。他们还鼓励使用“全团队”方法。这对测试人员意味着什么?本章讨论了一些涉及团队结构和物流的问题。创建一个有凝聚力的团队不仅仅是移动椅子和桌子。

Agile teams stress that face-to-face communication is critical to the success of a project. They also encourage using the “whole-team” approach. What does this mean to the testers? This chapter talks about some of the issues involving team structure and physical logistics. There’s more to creating a cohesive team than just moving chairs and desks.

团队架构

Team Structure

拥有独立的职能部门会使敏捷团队的日子不好过。持续的沟通至关重要。团队成员需要彼此密切合作,无论工作是在虚拟环境中还是在同一地点完成。

Having separate functional groups can make life difficult for agile teams. Constant communication is critical. Team members need to work closely with one another, whether the work is done virtually or in the same physical location.

我们在这里互换使用术语“QA 团队”和“测试团队”。可以争论“QA 团队”是否真的在做质量保证,但这个术语已经成为测试团队的常用词,所以我们也使用它。

We use the terms “QA team” and “test team” interchangeably here. It can be argued whether “QA teams” are really doing quality assurance or not, but the term has become a common one attached to test teams, so we use it too.

独立的质量保证团队

Independent QA Teams

许多组织,无论大小,都认为拥有一个独立的 QA 或测试团队很重要,这样才能获得关于产品质量的诚实意见。我们经常被问到这样的问题,“在整个团队方法中是否有测试组织的位置?” 和“如果是这样,它的作用是什么?”

Many organizations, both large and small, think it is important to have an independent QA or test team in order to get an honest opinion about the quality of a product. We’re often asked the questions, “Is there a place for a test organization in the whole-team approach?” and “If so, what is its role?”

我们希望将 QA 团队与开发团队分开的一些原因是:

Some of the reasons we’re given for wanting to keep the QA team separate from the development team are:

图像拥有独立的检查和审计角色很重要。

It is important to have that independent check and audit role.

图像该团队可以提供与产品质量相关的公正和外部观点。

The team can provide an unbiased and outside view relating to the quality of the product.

图像如果测试人员与开发人员的合作过于紧密,他们将开始像开发人员一样思考,并失去他们的客户观点。

If testers work too closely with developers, they will start to think like developers and lose their customer viewpoint.

图像如果测试人员和开发人员向同一个人报告,则存在优先交付任何代码而不是交付测试代码的危险。

If the testers and developers report to the same person, there is a danger that the priority becomes delivering any code rather than delivering tested code.

团队经常将“独立”与“独立”混淆。如果报告结构、预算和流程保持在独立的功能区域,则程序员和测试人员之间的划分是不可避免的。这会导致摩擦、竞争和“我们对他们”的态度。时间浪费在重复的会议上,程序员和测试人员没有共同的目标,并且不存在信息共享。

Teams often confuse “independent” with “separate.” If the reporting structure, budgets, and processes are kept in discrete functional areas, a division between the programmers and testers is inevitable. This can lead to friction, competition, and an “us versus them” attitude. Time is wasted on duplicate meetings, programmers and testers don’t share a common goal, and information sharing is nonexistent.

有一个 QA 经理和一个独立的测试团队是有原因的。但是,我们建议更改原因和结构。与其让测试人员作为一个独立的团队在编码后测试应用程序,不如将团队视为一个测试人员社区。提供一个学习型组织来帮助您的测试人员进行职业发展,并提供一个分享想法和互相帮助的地方。如果 QA 经理成为组织中的实践领导者,该人将能够教授测试人员所需的技能,使他们变得更强大,能够更好地应对不断变化的环境。

There are reasons for having a QA manager and an independent test team. However, we suggest changing the reasons as well as the structure. Rather than keeping the testers separate as an independent team to test the application after coding, think about the team as a community of testers. Provide a learning organization to help your testers with career development and a place to share ideas and help each other. If the QA manager becomes a practice leader in the organization, that person will be able to teach the skills that testers need to become stronger and better able to cope with the ever-changing environment.

我们不认为将测试人员与项目团队集成会妨碍测试人员做好他们的工作。事实上,敏捷团队的测试人员对他们作为客户倡导者的角色感觉非常强烈,也觉得他们可以影响团队的其他成员进行质量思考。

We don’t believe that integrating the testers with the project teams prevents testers from doing their jobs well. In fact, testers on agile teams feel very strongly about their role as customer advocate and also feel they can influence the rest of the team in quality thinking.

将测试人员集成到敏捷项目中

Integration of Testers into an Agile Project

敏捷开发中的全团队方法已经促使许多采用敏捷开发的组织解散了他们独立的 QA 团队,并将他们的测试人员派往项目组工作。虽然这听起来不错,但一些组织发现它并没有按预期工作。当发现自己在一个敏捷开发团队中不知道自己应该做什么时,不止一个组织的大部分(如果不是全部)测试人员都辞职了。

The whole-team approach in agile development has provoked many organizations that have adopted agile development to disband their independent QA teams and send their testers to work with the project groups. While this sounds great, some organizations have found that it doesn’t work as expected. More than one organization has had most, if not all, of their testers quit when they found themselves on an agile development team with no idea what they should be doing.

开发人员接受结对编程、测试驱动开发和其他敏捷实践方面的培训,而测试人员通常似乎根本没有接受任何培训。许多组织未能认识到测试人员还需要接受结对测试、处理不完整和不断变化的需求、自动化以及所有其他所需的新技能方面的培训。测试人员接受培训和指导非常重要,这样他们可以获得有助于他们成功的技能和理解,例如如何与客户合作编写面向业务的测试。程序员可能还需要指导以了解面向业务的测试的重要性以及整个团队编写和自动化测试的方法。

Developers get training on pair programming, test-driven development, and other agile practices, while testers often seem to get no training at all. Many organizations fail to recognize that testers also need training on pair testing, working with incomplete and changing requirements, automation, and all of the other new skills that are required. It’s critical that testers receive training and coaching so that they can acquire the skills and understanding that will help them succeed, such as how to work with customers to write business-facing tests. Programmers also might need coaching to understand the importance of business-facing tests and the whole-team approach to writing and automating tests.

Janet 帮助将几个独立的测试团队集成到敏捷项目中。她发现大多数测试人员可能需要长达六个月的时间才能开始对使用新流程充满信心。

Janet has helped integrate several independent test teams into agile projects. She finds that it can take up to six months for most testers to start feeling confident about working with the new process.

程序员和测试人员的配对只能改善关于产品质量的沟通。如果无法在开发环境中重现该行为,开发人员通常需要在测试人员的工作站上观察应用程序的行为。测试人员有时可以与开发人员坐下来重现问题,这比他们尝试记录缺陷中的步骤更容易、更快速。这种互动减少了花在非口头交流上的时间。

The pairing of programmers and testers can only improve communication about the quality of the product. Developers often need to observe the behavior of the application on the tester’s workstation if that behavior can’t be reproduced in the development environment. Testers can sometimes sit down with the developer to reproduce a problem more easily and quickly than they can by trying to record the steps in a defect. This interaction reduces the time spent on non-oral communication.

我们从测试人员那里听到的关于此主题的评论包括以下内容:

Comments we’ve heard from testers on this subject include the following:

图像“更接近产品的开发让我成为更好的测试人员。”

“Being closer to the development of the product makes me a better tester.”

图像“与开发人员共进午餐可以建立一个更好的团队,一个希望并喜欢一起工作的团队。”

“Going to lunch with developers builds a better team, one that wants and likes to work together.”

集成项目团队的一大优势是只有一个预算和一个时间表。如果所有功能都没有完成,就没有“测试”时间可以缩短。如果没有时间测试新功能,那么一开始就没有时间开发它。正如我们在整本书中指出的那样,整个团队对质量负责的方法非常强大。

One major advantage of an integrated project team is that there’s only one budget and one schedule. There is no “testing” time to cut if all of the functionality is not finished. If there is no time to test a new feature, then there is no time to develop it in the first place. The whole-team approach to taking responsibility for quality is very powerful, as we point out throughout this book.

丽莎的故事

Lisa’s Story

我曾经加入过一个 XP 团队,该团队一直完全依赖于单元级测试,并且之前从未有人担任过测试员角色。他们的客户对结果不太满意,因此他们决定聘请一名测试员。当我参加每日站立会议时,我不被允许谈论测试任务。测试时间不包括在故事估算中,测试任务也不是迭代计划的一部分。一旦编码任务完成,故事就会被标记为“完成”。

I once joined an XP team that had been depending solely on unit-level testing and had never had anyone in a tester role before. Their customer wasn’t all that happy with the results, so they decided to hire a tester. While I attended daily stand-ups, I wasn’t allowed to talk about testing tasks. Testing time wasn’t included in story estimates, and testing tasks weren’t part of iteration planning. Stories were marked “done” as soon as coding tasks were complete.

在团队错过了计划在三个两周迭代后发布的发布日期后,我要求团队的教练尝试全团队测试方法。测试任务与编码任务一起出现在董事会上。在测试任务完成之前,故事不再被视为完成。程序员承担了测试任务,而我是日常站立会议的完整参与者。该团队在满足他们设定的发布计划方面没有更多问题。

After the team missed the release date, which was planned for after three two-week iterations, I asked the team’s coach to try the whole-team approach to testing. Testing tasks went up on the board along with coding tasks. Stories were no longer considered done until testing tasks were finished. Programmers took on testing tasks, and I was a full participant in daily stand-ups. The team had no more issues meeting the release plans they set.

——丽莎

—Lisa

测试人员需要是开发团队的正式成员,测试任务需要与其他任务一样受到重视。同样,整个团队的测试方法对确保在每次迭代和发布结束时完成测试任务大有帮助。一定要使用回顾来评估测试人员需要与他们的新敏捷团队集成的内容以及他们可能需要获得的技能。例如,测试人员可能需要来自程序员或特定类型测试专家的更多支持。

Testers need to be full-fledged members of the development team, and testing tasks need to be given the same attention as other tasks. Again, the whole-team approach to testing goes a long way toward ensuring that testing tasks are completed by the end of each iteration and release. Be sure to use retrospectives to evaluate what testers need to integrate with their new agile team and what skills they might need to acquire. For example, testers might need more support from programmers, or from someone who’s an expert in a particular type of testing.

为敏捷开发规划组织变革的明智方法对成功转型至关重要。让 QA 和开发经理弄清楚他们自己在新的敏捷组织中的角色。让他们计划如何帮助他们的测试人员和开发人员在新的敏捷团队中提高工作效率。提供团队不知道的敏捷实践培训。确保所有团队都可以相互沟通。提供一个框架,让每个团队边走边学,团队将找到成功的方法。

A smart approach to planning the organizational changes for agile development makes all the difference to a successful transition. Ask the QA and development managers to figure out their own roles in the new agile organization. Let them plan how they will help their testers and developers be productive on the new agile teams. Provide training in agile practices that the team doesn’t know. Make sure all of the teams can communicate with each other. Provide a framework that lets each team learn as it goes, and the teams will find a way to succeed.

请参阅参考书目以获取 Christophe 关于管理敏捷团队的一些著作的链接。

See the bibliography for a link to some of Christophe’s writings on managing agile teams.

敏捷项目团队

Agile Project Teams

敏捷项目团队通常被认为是跨职能的,因为每个团队都有来自许多不同背景的成员。传统的跨职能团队和敏捷团队之间的区别在于整个团队努力的方法。只要项目或永久团队存在,成员就不仅仅是“代表”他们在团队中的职能,而是成为团队的真正成员(见图4-1)。

Agile project teams are generally considered cross-functional, because each team has members from many different backgrounds. The difference between a traditional cross-functional team and an agile team is the approach to the whole-team effort. Members are not just “representing” their functions in the team but are becoming true members of the team for as long as the project or permanent team exists (see Figure 4-1).

图 4-1传统职能团队结构与敏捷团队结构

Figure 4-1 Traditional functional teams structure vs. agile team structure

图像

由于项目规模不同,项目团队的结构也可能不同。拥有大型项目或许多同时发生的项目的组织正在使用矩阵型结构取得成功。来自不同职能领域的人联合起来组成一个虚拟团队,同时仍然向各自的组织结构汇报。在大型组织中,一群测试人员可能会从一个项目转移到另一个项目。一些专家,例如安全或性能测试人员,可能会在多个团队之间共享。如果您正在启动一个项目,请确定该项目将需要的所有资源。在开始之前确定所需的测试人员数量和所需的技能组合。测试人员从团队开始,一直工作到项目完成,然后再继续下一个项目。

Because projects vary in size, project teams might vary in structure. Organizations with large projects or many projects that happen simultaneously are having success using a matrix-type structure. People from different functional areas combine to form a virtual team while still reporting back to their individual organizational structures. In a large organization, a pool of testers might move from project to project. Some specialists, such as security or performance testers, might be shared among several teams. If you’re starting up a project, identify all of the resources the project will need. Determine the number of testers required and the skill set needed before you start. The testers start with the team and keep working until the project is complete, and at that time they go on to the next project.

虽然测试人员是团队的一部分,但他们的日常工作与项目团队的其他工作一样受到管理。测试人员可以从更大的测试人员社区中汲取新想法,其中包括大型组织中不同项目团队的测试人员。所有测试人员都可以共享知识和想法。在实施绩效审查的组织中,QA 经理(如果有的话)可能会推动审查并从项目团队那里获得输入。

While testers are part of the team, their day-to-day work is managed the same as the rest of the project team’s work. A tester can bounce new ideas off of the larger tester community, which includes testers on different project teams across a large organization. All testers can share knowledge and ideas. In organizations that practice performance reviews, the QA manager (if there is one) might drive the reviews and get input from the project team.

与任何新团队一样,团队凝聚起来需要一段时间。如果项目的长度很短并且团队不断变化,组织需要意识到每个项目的第一次或两次迭代将包括新团队成员习惯彼此合作。根据需要重构您的组织,并记住将您的客户包括在内。最好的团队是那些学会一起工作并且彼此建立信任的团队。

As with any new team, it takes a while for a team to jell. If the length of the project is short and the teams are constantly changing, the organization needs to be aware that the first iteration or two of every project will include the new team members getting used to working with each other. Refactor your organization as needed, and remember to include your customers. The best teams are those that have learned to work together and have developed trust with one another.

实体物流

Physical Logistics

许多正在考虑采用敏捷的组织都试图在不将团队置于开放式环境中的情况下创建项目团队。为了支持敏捷价值观和原则,当团队可以随时访问所有团队成员、轻松查看所有项目进度图表以及促进沟通的环境时,他们会更好地工作。

Many organizations that are thinking of adopting agile try to create project teams without co-locating the team in an open-plan environment. To support agile values and principles, teams work better when they have ready access to all team members, easy visibility of all project progress charts, and an environment that fosters communication.

靠近程序员的测试人员和客户可以实现必要的社区化。如果物流禁止同地办公,团队就可以发挥创造力。

Testers and customers sitting close to the programmers enable the necessary communalization. If logistics prohibit co-location, teams can be inventive.

珍妮特的故事

Janet’s Story

我在一个团队中工作,那里的空间阻止了所有团队成员坐在一起。程序员有一个可以轻松结对编程的区域,但测试人员和客户坐在另一个区域。起初,是测试人员前往故事板区域,程序员坐在那里参加站立会议,每当他们向其中一位程序员提出问题时。很少有程序员会前往(大约 50 英尺)测试人员区域。我开始在手边放一个糖果盘和零食,并鼓励开发人员想吃多少就吃多少。但有一个规则——如果他们是来拿糖果的,他们需要向其中一名测试员提问。随着时间的推移,所有团队成员的步行距离都变短了。没有一方走完所有的路,交流也很活跃。

I worked in a team where space prevented all team members from sitting together. The programmers had an area where they could pair-program with ease, but the testers and the customer were seated in another area. At first, it was the testers that made the trip to the storyboard area where the programmers sat to participate in stand-ups and whenever they had a question for one of the programmers. Few of the programmers made the trip (about 50 feet) to the testers’ area. I started keeping a candy dish handy with treats and encouraged the developers to take some as often as they wanted. But there was one rule—they needed to ask a question of one of the testers if they came for candy. Over time, the walk got shorter for all team members. No one side was doing all of the walking, and communication flourished.

——珍妮特

—Janet

团队规模为组织提供了不同类型的挑战。小团队意味着小区域,因此通常更容易将成员放在一起。大型团队可能分布在全球范围内,因此需要虚拟通信工具。将大型团队放在同一地点通常意味着翻新现有空间,而有些组织不愿意这样做。了解您的约束,并尝试为您的团队遇到的问题找到解决方案,而不是仅仅接受事物的“现状”。

Team size offers different types of challenges to the organization. Small teams mean small areas, so it is usually easier to co-locate members. Large teams might be spread globally, and virtual communication tools are needed. Co-locating large teams usually means renovating existing space, which some organizations are reluctant to do. Understand your constraints, and try to find solutions to the problems your team encounters rather than just accepting things as “the way it is.”

珍妮特的故事

Janet’s Story

我工作的一个团队从场地的一个角落开始,但在三年的时间里扩大了规模,逐渐占据了场地的 70% 以上。墙壁被拆除,办公室被拆除,并创建了大片开放区域。团队的开放区域和豆荚运作良好,但所有开放空间都意味着墙壁空间丢失。Windows 变成了故事板和白板,并且订购了滚动白板,可以在团队需要时使用。

One team I worked on started in one corner of the floor, but expanded over the course of three years, gradually taking over 70% of the floor. Walls were taken down, offices removed, and large open areas created. The open areas and pods of teams worked well, but all the open space meant wall space was lost. Windows became story boards and whiteboards, and rolling whiteboards were ordered that could be used as teams needed them.

——珍妮特

—Janet

位于同一地点的团队并不总是生活在一个完美的世界中,分布式团队面临着另一组挑战。分布式团队需要技术来帮助他们沟通和协作。电话会议、视频会议、网络摄像头和即时消息是一些可以促进多个位置的团队进行实时协作的工具。

Co-located teams don’t always live in a perfect world, and distributed teams have a another set of challenges. Distributed teams need technology to help them communicate and collaborate. Teleconferencing, video conferencing, webcams, and instant messaging are some tools that can promote real-time collaboration for teams in multiple locations.

无论团队是同地还是分布式,通常都会出现关于敏捷团队需要哪些资源以及如何获取这些资源的相同问题。我们将在下一节中讨论这些。

Whether teams are co-located or distributed, the same questions usually come up about what resources are needed on an agile team and how to obtain them. We’ll discuss these in the next section.

资源

Resources

新的敏捷团队成员和他们的经理对团队的构成有很多疑问。我们可以使用与传统项目相同的测试人员,还是需要雇用不同类型的测试人员?我们需要多少测试人员?我们需要具有其他专业技能的人吗?在本节中,我们将稍微讨论一下这些问题。

New agile team members and their managers have lots of questions about the makeup of the team. Can we use the same testers that we had with our traditional projects, or do we need to hire a different type of tester? How many testers will we need? Do we need people with other specialized skills? In this section, we talk a little about these questions.

测试人员-开发人员比率

Tester-Developer Ratio

关于测试人员数量与开发人员数量的“正确”比例,已经有很多讨论。组织已使用此比率来确定项目需要多少测试人员,以便他们可以相应地雇用。与传统项目一样,没有“正确”的比例,每个项目都需要单独评估。所需测试人员的数量会有所不同,具体取决于应用程序的复杂性、测试人员的技能组合以及所使用的工具。

There have been many discussions about the “right” ratio of the number of testers to the number of developers. This ratio has been used by organizations to determine how many testers are needed for a project so that they can hire accordingly. As with traditional projects, there is no “right” ratio, and each project needs to be evaluated on its own. The number of testers needed will vary and depends upon the complexity of the application, the skill set of the testers, and the tools used.

我们曾在测试人员与开发人员比例为 1:20 到 1:1 的团队中工作。这是我们的一些经验。

We have worked on teams with a tester-developer ratio of anywhere from 1:20 to 1:1. Here are a couple of our experiences.

珍妮特的故事

Janet’s Story

我参与了一个比例为 1:10 的项目,该项目开发了一个消息处理系统。GUI 非常少,我手动测试了应用程序的那部分,查看可用性以及它与客户期望的匹配程度。当我与程序员一起工作以验证所编写的测试用例的有效性时,程序员进行了所有的自动化回归测试。我与开发人员对故事进行了配对测试,包括对特定故事进行负载测试。

I worked on a project with a 1:10 ratio that developed a message-handling system. There was very little GUI, and I manually tested that part of the application, looking at usability and how well it matched the customer’s expectations. The programmers did all of the automated regression testing while I worked with them to validate the effectiveness of the test cases written. I pair-tested stories with the developers, including load testing specific stories.

我从来没有觉得我没有足够的时间来做我需要做的测试,因为开发人员认为质量是整个团队的责任。

I never felt that I didn’t have enough time to do the testing I needed to, because the developers believed that quality was the whole team’s responsibility.

——珍妮特

—Janet

丽莎的故事

Lisa’s Story

我曾经是一个由多达 20 名程序员组成的团队中唯一的专业测试员,该团队正在开发一个 Internet 购物网站的内容管理系统。当程序员同时负责手动测试和测试自动化时,团队开始变得真正富有成效。一两个程序员在每次迭代中都戴着“测试员的帽子”,在编码和执行手动测试之前编写面向客户的测试。其他程序员在迭代期间接手了测试自动化任务。

I was once the only professional tester on a team of up to 20 programmers developing a content management system on an Internet shopping website. The team began to get really productive when the programmers took on responsibility for both manual testing and test automation. One or two programmers wore a “tester hat” for each iteration, writing customer-facing tests ahead of coding and performing manual tests. Additional programmers picked up the test automation tasks during the iteration.

相反,我现在的团队每三到五个程序员就有两个测试人员。我们制作的基于 Web 的金融应用程序具有高度复杂的业务逻辑、高风险和测试密集型。测试任务加起来通常与编程任务的时间相同。即使测试人员与程序员的比例相对较高,程序员也会完成大部分功能测试自动化,有时还会接手手动测试任务。专门的测试任务,例如编写高级测试用例和详细的面向客户的测试,通常由测试人员完成。

Conversely, my current team has had two testers for every three to five programmers. The web-based financial application we produce has highly complex business logic, is high risk, and test intensive. Testing tasks often add up to the same amount of time as programming tasks. Even with a relatively high tester–programmer ratio, programmers do much of the functional test automation and sometimes pick up manual testing tasks. Specialized testing tasks such as writing high-level test cases and detailed customer-facing tests are usually done by the testers.

——丽莎

—Lisa

团队不应关注比率,而应评估他们需要的测试技能并找到合适的资源。负责测试的团队可以不断评估自己是否具备所需的专业知识和带宽。使用回顾来确定是否存在雇佣更多测试人员可以解决的问题。

Rather than focus on a ratio, teams should evaluate the testing skills they need and find the appropriate resources. A team that takes responsibility for testing can continually evaluate whether it has the expertise and bandwidth it needs. Use retrospectives to identify whether there’s a problem that hiring more testers would solve.

雇用敏捷测试人员

Hiring an Agile Tester

正如我们在第 2 章敏捷测试人员的十项原则”中所讨论的,有一些品质可以使测试人员适合在敏捷团队中工作。我们不想详细说明要聘请什么样的测试人员,因为每个团队的需求都不同。但是,我们确实认为态度是一个重要因素。这是一个关于 Lisa 的团队如何努力聘请新的敏捷测试人员的故事。

As we discussed in Chapter 2, “Ten Principles for Agile Testers,” there are certain qualities that make a tester suited to working on an agile team. We don’t want to go into a lot of detail about what kind of tester to hire, because every team’s need is different. However, we do believe that attitude is an important factor. Here’s a story of how Lisa’s team struggled to hire a new agile tester.

丽莎的故事

Lisa’s Story

我们第一次尝试招募另一名测试员时并不十分成功。第一份招聘启事引起了很多反响,我们面试了三名候选人,但没有找到合适的人选。程序员需要一个“技术人员”,但我们也需要一个有能力与业务人员合作并帮助他们生成示例和需求的人。我们努力确定职位发布的内容,以吸引具有正确态度和心态的候选人。

Our first attempt at recruiting another tester was not very successful. The first job posting elicited many responses, and we interviewed three candidates without finding a good fit. The programmers wanted someone “techie,” but we also needed someone with the skills to collaborate with business people and help them to produce examples and requirements. We struggled to determine the content of the job posting in order to attract candidates with the right attitude and mind-set.

在征求 Janet 和敏捷测试社区其他同事的意见和建议后,我们决定寻找具有第 2中描述的思维定势的测试人员。我们更改了职位发布以包括以下项目:

After soliciting opinions and suggestions from Janet and other colleagues in the agile testing community, we decided to look for a tester with the mind-set that is described in Chapter 2. We changed the job posting to include items such as these:

• 拥有编写黑盒和 GUI 测试用例、设计测试以降低风险以及帮助业务专家定义需求的经验

• Experience writing black box and GUI test cases, designing tests to mitigate risks, and helping business experts define requirements

• 具有编写简单的 SQL 查询和插入/更新语句的经验,并且基本掌握 Oracle 或其他关系数据库

• Experience writing simple SQL queries and insert/update statements and basic grasp of Oracle or another relational database

• 至少一年使用某些脚本或编程语言和/或开源测试工具的经验

• At least one year of experience with some scripting or programming language and/or open source test tools

• 能够使用基本的 Unix 命令

• Ability to use basic Unix commands

• 与程序员和业务专家合作的经验

• Experience collaborating with programmers and business experts

• 有基于上下文、探索性或情景测试的经验者优先

• Experience in context-based, exploratory, or scenario testing a plus

• 能够作为自组织团队的一部分工作,在这个团队中您每天与同事协调确定您的任务,而不是等待分配给您的工作

• Ability to work as part of a self-organizing team in which you determine your tasks on a daily basis in coordination with coworkers rather than waiting for work to be assigned to you

这些要求使候选人更适合敏捷测试工作。我仔细地进行了筛选,排除了具有“素质警察”心态的人。追求专业发展并对敏捷开发表现出兴趣的测试人员更有可能拥有正确的心态。团队需要在测试工具和自动化领域有实力的人,因此对学习的热情是最重要的。

These requirements brought candidates more suited to an agile testing job. I proceeded carefully with screening, ruling out people with a “quality police” mentality. Testers who pursued professional development and showed interest in agile development were more likely to have the right mind-set. The team needed someone who would be strong in the area of test tools and automation, so a passion for learning was paramount.

这种更具创造性的招募测试人员的方法得到了回报。那时,要找到优秀的“敏捷测试员”候选人并不容易,但后来的寻找更加顺利。我们发现,将测试员职位发布在不太明显的地方,例如 Ruby 邮件列表或本地敏捷用户组,有助于接触到更广泛的合适候选人。

This more creative approach to recruiting a tester paid off. At that time, it wasn’t easy to find good “agile tester” candidates, but subsequent searches went more smoothly. We found that posting the tester position in less obvious places, such as a Ruby mailing list or the local agile user group, helped reach a wider range of suitable candidates.

聘请一名敏捷测试人员教会了我很多关于敏捷测试思维方式​​的知识。有些测试人员拥有非常好的技能,他们对任何传统测试团队都很有价值,但由于他们对测试的态度,他们不适合敏捷团队。

Hiring an agile tester taught me a lot about the agile testing mind-set. There are testers with very good skill sets who would be valuable to any traditional test team but would not be a good fit on an agile team because of their attitude toward testing.

——丽莎

—Lisa

我们需要考虑的不仅仅是测试人员和程序员在团队中扮演的角色。无论您想担任什么角色,最重要的考虑因素是该人如何融入您的团队。使用敏捷的全团队方法,团队中的专家可能会被要求走出他们的专业领域并参与其他活动。每个团队成员都需要高度关注质量和交付业务价值。在扩大团队时,考虑的不仅仅是技术技能。

We need to consider more than just the roles that testers and programmers perform on the team. No matter what role you’re trying to fill, the most important consideration is how that person will fit on your team. With the agile whole-team approach, specialists on the team might be asked to step outside their areas of expertise and pitch in on other activities. Each team member needs to have a strong focus on quality and delivering business value. Consider more than just technical skills when you’re expanding your team.

建立团队

Building a Team

我们已经就整个团队的方法谈了很多。但这样的变化并不仅仅发生。我们被问到这样的问题,“我们如何让团队凝聚起来?” 或“我们如何促进整个团队的方法?” 其中一个重要问题是:“我们如何让每个人都保持积极性并专注于实现商业价值的目标?”

We’ve talked a lot about the whole-team approach. But changes like that don’t just happen. We get asked questions like, “How do we get the team to jell?” or “How do we promote the whole-team approach?” One of the big ones is: “How do we keep everyone motivated and focused on the goal of delivering business value?”

自组织团队

Self-Organizing Team

根据我们的经验,当团队能够识别和解决他们自己的问题时,他们会取得最好的进展。如果您是一名经理,请抵制将您所有的好想法强加给团队的诱惑。有些问题,比如人事问题,最好由管理者来解决,而在需要领导的时候,也有教练需要大力鼓励和带领球队的时候。一个新的敏捷团队需要时间来学习如何确定问题的优先级和解决问题,但团队犯错误和跌跌撞撞几次也没关系。我们认为一个高效的团队必须自我成长。如果您是一名测试人员,您可以很好地帮助团队找出获得快速反馈的方法,使用回顾等实践来确定问题的优先级并解决问题,并找到帮助您的团队开发出更好软件的技术。

In our experience, teams make the best progress when they’re empowered to identify and solve their own problems. If you’re a manager, resist the temptation to impose all your good ideas on the team. There are problems, such as personnel issues, that are best solved by managers, and there are times a coach needs to provide strong encouragement and lead the team when it needs leadership. It takes time for a new agile team to learn how to prioritize and solve its problems, but it’s okay for the team to make mistakes and stumble a few times. We think a high-functioning team has to grow itself. If you’re a tester, you’re in a good position to help the team figure out ways to get fast feedback, use practices such as retrospectives to prioritize and address issues, and find the techniques that help your team produce better software.

涉及其他团队

Involving Other Teams

您可能需要让其他团队参与进来,以帮助您的团队取得成功。安排会议;找到尽可能多的沟通方式。使用 Scrum of Scrums 来保持多个团队的协调,或者只是让其他团队参与进来。如果您必须聘请专家来帮助进行安全测试,请与该专家结对并尽可能多地学习,并帮助他们了解您的项目。

You might need to get other teams on board to help your team succeed. Set up meetings; find ways to communicate as much as possible. Use a Scrum of Scrums to keep multiple teams coordinated, or just get involved with the other teams. If you have to bring in an expert to help with security testing, pair with that expert and learn as much as you can, and help them learn about your project.

如果团队分散在不同的地点和时区,想办法尽可能多地进行直接沟通。也许代表来自每个团队可以每周调整一到两次他们的时间,这样他们就可以每周召开一次电话会议。尽可能打电话而不是发送电子邮件。Lisa 的团队调整了其计划会议时间,以包括一名在深夜工作的远程团队成员。他们将会议安排在他的一天与团队一天的其余时间重叠的时间。

If teams are scattered in different locations and time zones, figure out how to get as much direct communication as possible. Maybe representatives from each team can adjust their hours once or twice a week so that they can teleconference once a week. Make a phone call instead of sending an email whenever possible. Lisa’s team adjusted its planning meeting times to include a remote team member who works late at night. They schedule meetings for a time where his day overlaps with the rest of the team’s day.

第 9 章支持团队的面向业务的测试工具包”给出了帮助远程团队协作的工具示例。

Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” gives examples of tools that help remote teams collaborate.

每个团队成员都具有同等价值

Every Team Member Has Equal Value

每个团队成员对团队都具有同等的价值。如果测试人员或任何其他团队成员感到被冷落或被低估,则整个团队的方法注定要失败。确保邀请测试人员参加所有会议。如果您是一名测试员,而有人忘记邀请您参加会议,请自己邀请。非技术测试人员可能认为他们会在设计会议上格格不入或不知所措,但有时他们会提出技术人员没有想到的好问题。

Every team member has equal value to the team. If testers or any other team members feel left out or less valued, the whole-team approach is doomed. Make sure testers are invited to all meetings. If you’re a tester and someone forgets to invite you to a meeting, invite yourself. Nontechnical testers might think they’ll be out of place or overwhelmed at a design meeting, but sometimes they ask good questions that the techies didn’t think of.

测试人员有权寻求和获得帮助。如果您是一名被自动化问题困住的测试人员,请鼓起勇气向团队成员寻求帮助。该人现在可能很忙,但他或她必须承诺在合理的时间内为您提供帮助。如果您是团队的经理或领导者,请确保这种情况正在发生,如果没有发生,请向团队提出问题。

Testers have a right to ask for and get help. If you’re a tester stuck on an automation problem, have the courage to ask a team member for help. That person might be busy right now, but he or she must commit to helping you in a reasonable amount of time. If you’re a manager or leader on your team, make sure this is happening, and raise the issue to the team if it’s not.

绩效和奖励

Performance and Rewards

以个人为基础衡量和评估绩效可能会破坏团队协作。我们不希望程序员觉得她不应该承担测试任务,因为她被评为交付生产代码。我们不希望系统管理员忙于确保实现她的个人目标,以致于无法帮助解决测试环境问题。

Measuring and rating performance on an individual basis risks undermining team collaboration. We don’t want a programmer to feel she shouldn’t take on a testing task because she’s rated on delivering production code. We don’t want a system administrator to be so busy making sure her individual goals are met that she can’t help with a test environment problem.

相反,一个试图与团队良好合作的优秀执行者不应该因为团队的其他成员没有齐心协力而被淘汰。这是经理需要挺身而出帮助团队找到方向的时候。如果主要错误进入生产,没有人应该责怪测试人员。相反,整个团队应该分析发生的事情并开始采取措施防止再次发生。

Conversely, a good performer who was trying to work well with the team shouldn’t be knocked because the rest of the team didn’t pull together. This is a time when a manager needs to step up and help the team find its way. If major bugs made it to production, nobody should blame the testers. Instead, the whole team should analyze what happened and start taking steps to prevent a recurrence.

开发团队需要牢记业务需求。设定服务于业务的目标,提高盈利能力,让客户更快乐。与企业密切合作,让您的成功帮助整个公司取得成功。

The development team needs to keep the business needs in mind. Set goals that serve the business, increase profitability, and make the customers happier. Work closely with the business so that your successes help the whole company succeed.

正如我们在第 3 章文化挑战”中提到的,庆祝每一次成功,无论多么小。庆祝活动可能是击掌、公司提供的午餐,或者可能只是提前下班参加一些社交活动。Lisa 团队的 ScrumMaster 在站立会议上为特殊成就颁发金星。感谢帮助您和您的团队的人。

As we mentioned in Chapter 3, “Cultural Challenges,” celebrate every success, however small. A celebration might be a high-five, a company-provided lunch, or maybe just leaving work early to socialize a bit. The ScrumMaster on Lisa’s team hands out gold stars at stand-up meetings for special accomplishments. Acknowledge the people who help you and your team.

团队可以找到新颖的方式来认可彼此的贡献。开发团队和客户团队都出席的迭代审查和演示会议是认可个人和团队成就的良好场所。

Teams can find novel ways to recognize each other’s contributions. Iteration review and demonstration meetings, where both the development team and customer team are present, are a good setting for recognizing both individual and team achievements.

阅读第 19 章结束迭代”中的“Shout-Out Shoebox ”想法。

Read about the “Shout-Out Shoebox” idea in Chapter 19, “Wrap Up the Iteration.”

你能做什么?

What Can You Do?

如果您是敏捷团队的新测试人员,尤其是新的敏捷团队,您可以做些什么来帮助团队克服组织挑战并取得成功?您如何融入团队并贡献您的特殊技能和经验?

If you’re a new tester on an agile team, especially a new agile team, what can you do to help the team overcome organizational challenges and succeed? How can you fit in with the team and contribute your particular skills and experience?

将我们在第 2 章中描述的十项原则付诸实践。勇气尤为重要。起身去与人交谈;询问你能提供什么帮助。通过直接沟通联系团队成员和其他团队。注意障碍并要求团队帮助移除它们。

Put the ten principles we described in Chapter 2 to work. Courage is especially important. Get up and go talk to people; ask how you can help. Reach out to team members and other teams with direct communication. Notice impediments and ask the team to help remove them.

敏捷开发之所以有效,是因为它消除了我们前进道路上的障碍,让我们做到最好。无论是个人还是团队,我们都可以感到自豪和满足。当我们遵循敏捷原则时,我们可以很好地协作,使用反馈来帮助改进我们的工作方式,并始终寻找新的更好的方法来实现我们的目标。所有这些意味着我们可以不断提高产品质量。

Agile development works because it gets obstacles out of our path and lets us do our best work. We can feel proud and satisfied, individually and as a team. When we follow agile principles, we collaborate well, use feedback to help improve how we work, and always look for new and better ways to accomplish our goals. All this means we can continually improve the quality of our product.

概括

Summary

在本章中,我们研究了为成功的敏捷测试和开发构建团队和结构的方法。

In this chapter, we looked at ways to build a team and a structure for successful agile testing and development.

图像考虑团队结构的重要性;虽然测试人员可能需要独立的思维模式,但将他们放在一个单独的团队中可能会适得其反。

Consider the importance of team structure; while testers might need an independent mind-set, putting them on a separate team can be counterproductive.

图像测试人员需要访问更大的测试人员社区来学习和尝试新想法。QA 团队可能能够在他们的组织内创建这个社区。

Testers need access to a larger community of testers for learning and trying out new ideas. QA teams might be able to create this community within their organization.

图像 整个团队聚在一起以促进协作很重要;如果团队是分布式的,提供促进沟通的工具。

It is important for the whole team to be located together, to foster collaboration; if the team is distributed, provide tools to promote communication.

图像雇用态度。

Hire for attitude.

图像没有正确的测试人员-开发人员比例。正确的答案是,“这取决于你的情况。”

There is no right tester–developer ratio. The right answer is, “It depends on your situation.”

图像团队需要自我组织,识别并找到解决自身问题的方法,并寻找改进方法。他们等不及有人告诉他们该怎么做。

Teams need to self-organize, identify and find solutions to their own problems, and look for ways to improve. They can’t wait for someone to tell them what to do.

图像管理层应该以促进团队努力实现业务价值的方式来奖励绩效,而不是在团队苦苦挣扎时惩罚良好的个人绩效。

Management should reward performance in a way that promotes the team’s effort to deliver business value but not penalize good individual performance if the team is struggling.

图像测试人员可以使用敏捷原则来提高自己的技能并增加他们对团队的价值。他们需要积极主动并找到可以做出贡献的方式。

Testers can use agile principles to improve their own skills and increase their value to the team. They need to be proactive and find ways that they can contribute.

第 5 章转换典型流程

Chapter 5 Transitioning Typical Processes

图像

传统项目中有许多流程不能很好地过渡到敏捷,因为它们需要重量级文档,或者是分阶段和封闭流程的固有部分,并且需要在每个阶段结束时签字。

There are many processes in a traditional project that don’t transition well to agile because they require heavyweight documentation or are an inherent part of the phased and gated process and require sign-offs at the end of each stage.

与其他任何事情一样,没有硬性规定可以将您的流程转换为更敏捷或更轻量级的流程。在本章中,我们将讨论其中的一些流程,并为您提供备选方案和指导,指导您如何在敏捷项目中使用它们。您将在第 III 部分第 IV部分和第 V 部分中找到有关这些备选方案的更多示例和详细信息。

Like anything else, there are no hard and fast rules for transitioning your processes to a more agile or lightweight process. In this chapter, we discuss a few of those processes, and give you alternatives and guidance on how to work with them in an agile project. You’ll find more examples and details about these alternatives in Parts III, IV, and V.

寻求轻量级流程

Seeking Lightweight Processes

当团队学习如何使用敏捷流程时,一些更传统的流程​​可能会在混乱中丢失。大多数习惯于工作的测试人员使用传统的分阶段和门控开发方法的人习惯于生成和使用指标,在正式的缺陷跟踪系统中记录缺陷,并编写详细的测试计划。这些在敏捷开发中适合什么地方?

When teams are learning how to use agile processes, some of the more traditional processes can be lost in the shuffle. Most testers who are used to working with traditional phased and gated development methodologies are accustomed to producing and using metrics, recording defects in a formal defect tracking system, and writing detailed test plans. Where do those fit in agile development?

许多软件组织必须遵守审计系统或质量过程模型。这些要求通常不会因为您开始使用敏捷开发实践而消失。事实上,有人担心敏捷开发会与CMMI、ISO 9000等模型和标准不兼容。

Many software organizations must comply with audit systems or quality process models. Those requirements don’t usually disappear just because you start using agile development practices. In fact, some people worry that agile development will be incompatible with such models and standards as CMMI and ISO 9000.

在敏捷项目上进行测试时,谈论所有新的和不同的东西可能会更有趣,但我们仍然需要衡量进度、跟踪缺陷和计划测试的方法。我们还需要准备好使用我们组织的质量模型。关键是让这些流程足够轻便,以帮助我们及时交付价值。让我们从查看指标开始。

It might be more fun to talk about everything that’s new and different when testing on an agile project, but we still need ways to measure progress, track defects, and plan testing. We also need to be prepared to work with our organization’s quality models. The key is to keep these processes lightweight enough to help us deliver value in a timely manner. Let’s start by looking at metrics.

指标

Metrics

指标可能会引起争议,我们花了很多时间来讨论它们。指标可能是一个浪费精力的坑,为了数字而数字。它们有时会以有害的方式使用,尽管它们不一定是坏的。他们可以指导您的团队并帮助它衡量您的团队在实现目标方面的进展。让我们来看看如何使用指标来帮助敏捷测试人员及其团队。

Metrics can be controversial, and we spend a lot of time talking about them. Metrics can be a pit of wasted effort, numbers for the sake of numbers. They are sometimes used in harmful ways, although they don’t have to be bad. They can guide your team and help it to measure your team’s progress toward its goals. Let’s take a look at how to use metrics to help agile testers and their teams.

精益测量

Lean Measurements

精益软件开发从业者寻找减少测量数量的方法,并找到将驱动正确行为的测量。实施精益软件开发:从概念到现金,作者 Mary 和 Tom Poppendieck,是一个很好的资源,可以教授如何将精益计划应用于您的测试和开发工作。

Lean software development practitioners look for ways to reduce the number of measurements and find measurements that will drive the right behaviors. Implementing Lean Software Development: From Concept to Cash, by Mary and Tom Poppendieck, is an excellent resource that teaches how to apply lean initiatives to your testing and development efforts.

根据 Poppendiecks [2007] 的说法,一个基本的精益衡量标准是“从概念到现金”,从客户的功能请求到交付软件所花费的时间。他们称这种测量为“周期时间”。重点在于团队“反复且可靠地”交付新业务价值的能力。然后团队尝试不断改进他们的流程并减少周期时间。

According to the Poppendiecks [2007], a fundamental lean measurement is the time it takes to go “from concept to cash,” from a customer’s feature request to delivered software. They call this measurement “cycle time.” The focus is on the team’s ability to “repeatedly and reliably” deliver new business value. Then the team tries to continuously improve their process and reduce the cycle time.

涉及整个团队的周期时间等度量比仅限于孤立角色或团体。修复缺陷通常需要多长时间?团队可以做些什么来减少延迟,减少所花费的时间?这些类型的指标鼓励协作以进行改进。

Measurements such as cycle time that involve the whole team are more likely to drive you toward success than are measures confined to isolated roles or groups. How long does it usually take to fix a defect? What can the team do to reduce that latency, the amount of time it takes? These types of metrics encourage collaboration in order to make improvements.

Poppendiecks 在他们的书中解释的另一个精益衡量标准是财务回报。如果团队正在开发一种有利可图的产品,则需要了解如何才能实现最大利润。即使团队正在开发主要目标不是利润的内部软件或其他产品,它仍然需要查看 ROI 以确保它提供最佳价值。确定业务目标并找到衡量团队交付成果的方法。公司是否正在努力吸引新客户?跟踪新功能发布时有多少新帐户登录。

Another lean measurement the Poppendiecks explain in their book is financial return. If the team is developing a profitable product, it needs to understand how it can work to achieve the most profit. Even if the team is developing internal software or some other product whose main goal isn’t profit, it still needs to look at ROI to make sure it is delivering the best value. Identify the business goals and find ways to measure what the team delivers. Is the company trying to attract new customers? Keep track of how many new accounts sign on as new features are released.

精益开发寻找取悦客户的方法,这应该是所有软件开发的目标。Poppendiecks 举例说明了衡量客户是否满意的简单方法。

Lean development looks for ways to delight customers, which ought to be the goal for all software development. The Poppendiecks give examples of simple ways you can measure whether your customers are delighted.

我们喜欢精益指标,因为它们与我们提供商业价值的目标一致。为什么我们对指标感兴趣?我们将在下一节中进行介绍。

We like the lean metrics, because they’re congruent with our goal to deliver business value. Why are we interested in metrics at all? We’ll go into that in the next section.

为什么我们需要指标

Why We Need Metrics

收集和跟踪指标是有充分理由的。也有一些非常糟糕的。任何人都可能以糟糕的方式使用好的指标,例如将它们用作单个团队成员绩效评估的基础。然而,没有指标,你如何衡量你的进步?

There are good reasons to collect and track metrics. There are some really bad ones too. Anyone can use good metrics in terrible ways, such as using them as the basis for an individual team member’s performance evaluation. However, without metrics, how do you measure your progress?

当指标被用作路标时——告诉团队何时偏离轨道或提供反馈表明它在正确的轨道上——它们值得收集。我们的单元测试数量每天都在增加吗?为什么代码覆盖率从 75% 下降到 65%?这可能是一个很好的理由——也许我们摆脱了测试覆盖的未使用代码。指标可以提醒我们注意问题,但孤立地看,它们通常不会提供价值。

When metrics are used as guideposts—telling the team when it’s getting off track or providing feedback that it’s on the right track—they’re worth gathering. Is our number of unit tests going up every day? Why did the code coverage take a dive from 75% to 65%? It might have been a good reason—maybe we got rid of unused code that was covered by tests. Metrics can alert us to problems, but in isolation they don’t usually provide value.

衡量实现团队目标的旅程中的里程碑的指标很有用。如果我们的目标是将单元测试代码覆盖率提高 3%,我们可能会在每次签入时运行代码覆盖率,以确保我们没有懈怠单元测试。如果我们没有达到预期的改进,找出原因比抱怨我们的奖金因此减少了多少更重要。我们不应专注于个别测量,而应专注于目标和实现该目标的趋势。

Metrics that measure milestones along a journey to achieve team goals are useful. If our goal is to increase unit test code coverage by 3%, we might run the code coverage every time we check in to make sure we didn’t slack on unit tests. If we don’t achieve the desired improvement, it’s more important to figure out why than to lament whatever amount our bonus was reduced as a result. Rather than focus on individual measurements, we should focus on the goal and the trending toward reaching that goal.

指标帮助团队(包括客户)跟踪迭代和发布或史诗中的进度。如果我们正在使用燃尽图,并且我们正在燃烧而不是燃烧,这是停止的危险信号,看看发生了什么,并确保我们理解并解决问题。也许团队缺少有关故事的重要信息。指标(包括燃尽图)不应用作惩罚形式或责备来源。例如,诸如“为什么您的估计值太低?”之类的问题。或者“为什么你不能完成所有的故事?” 最好来自团队并表述为“为什么我们的估计如此低?” 和“为什么我们没有完成我们的故事?”

Metrics help the team, customers included, to track progress within the iteration and within the release or epic. If we’re using a burndown chart, and we’re burning up instead of down, that’s a red flag to stop, take a look at what’s happening, and make sure we understand and address the problems. Maybe the team lacked important information about a story. Metrics, including burndown charts, shouldn’t be used as a form of punishment or source of blame. For example, questions like “Why were your estimates too low?” or “Why can’t you finish all of the stories?” would be better coming from the team and phrased as “Why were our estimates so low?” and “Why didn’t we get our stories finished?”

如果使用得当,指标可以激励团队。Lisa 的团队跟踪每个构建中运行的单元测试的数量。重大里程碑——100 次测试、1000 次测试、3000 次测试——值得庆祝。每天增加那么多的单元测试是对开发和客户团队的一个很好的反馈。然而,重要的是要认识到数字本身没有任何意义。例如,测试可能写得不好,或者要有一个经过良好测试的产品,我们可能需要 10,000 个测试。数字不能孤立地发挥作用。

Metrics, used properly, can be motivating for a team. Lisa’s team tracks the number of unit tests run in each build. Big milestones—100 tests, 1000 tests, 3000 tests—are a reason to celebrate. Having that number of unit tests go up every day is a nice bit of feedback for the development and customer teams. However, it is important to recognize that the number itself means nothing. For example, the tests might be poorly written, or to have a well tested product, maybe we need 10,000 tests. Numbers don’t work in isolation.

丽莎的故事

Lisa’s Story

Pierre Veragen 告诉我他工作的一个团队对指标过敏。团队成员决定停止测量他们的测试覆盖了多少代码。当他们决定在六个月后再次测量时,他们惊奇地发现该比率从 40% 下降到 12%。

Pierre Veragen told me about a team he worked on that was allergic to metrics. The team members decided to stop measuring how much code their tests covered. When they decided to measure again after six months, they were stunned to discover the rate had dropped from 40% to 12%.

不使用正确的指标会让您付出多少代价?

How much is it costing you to not use the right metrics?

——丽莎

—Lisa

当您试图弄清楚要衡量什么时,首先要了解您要解决的问题是什么。当你知道问题陈述时,你可以设定一个目标。这些目标需要是可衡量的。“将 XYZ 应用程序的平均响应时间减少到 20 个并发用户的 1.5 秒”比“提高 XYZ 应用程序性能”效果更好。如果您的目标是可衡量的,那么您需要收集的衡量指标来跟踪指标将是显而易见的。

When you’re trying to figure out what to measure, first understand what problem you are trying to solve. When you know the problem statement, you can set a goal. These goals need to be measurable. “Reduce average response time on the XYZ application to 1.5 seconds with 20 concurrent users” works better than “Improve the XYZ application performance.” If your goals are measurable, the measurements you need to gather to track the metrics will be obvious.

请记住将指标用作激励力量,而不是用来打击团队的士气。这个智慧值得重复:关注目标,而不是指标。也许您没有使用正确的指标来衡量您是否实现了团队的目标,或者您可能没有在上下文中解释它们。缺陷报告数量的增加可能意味着团队在测试方面做得更好,而不是他们编写了更多有缺陷的代码。如果你的指标无法帮助您了解实现目标的进度,您可能有错误的指标。

Remember to use metrics as a motivating force and not for beating down a team’s morale. This wisdom bears repeating: Focus on the goal, not the metrics. Maybe you’re not using the right metrics to measure whether you’re achieving your team’s objectives, or perhaps you’re not interpreting them in context. An increased number of defect reports might mean the team is doing a better job of testing, not that they are writing more buggy code. If your metrics aren’t helping you to understand your progress toward your goal, you might have the wrong metrics.

不应该用指标做什么

What Not to Do with Metrics

马克吐温推广了他归功于本杰明迪斯雷利的一句话:“谎言分三种:谎言、该死的谎言和统计数据。” 可衡量的目标是一件好事;如果你不能以某种方式衡量它们,你就无法判断你是否实现了它们。另一方面,使用指标来判断个人或团队的表现是危险的。统计数据本身可以被扭曲成任何解释并以有害的方式使用。

Mark Twain popularized the saying, which he attributed to Benjamin Disraeli, “There are three kinds of lies: lies, damned lies, and statistics.” Measurable goals are a good thing; if you can’t gauge them in some way, you can’t tell if you achieved them. On the other hand, using metrics to judge individual or team performance is dangerous. Statistics by themselves can be twisted into any interpretation and used in detrimental ways.

以代码行为例,这是一种传统的软件量尺。更多的代码行是好事,意味着团队一直在生产,还是坏事,意味着团队正在编写低效的意大利面条式代码?

Take lines of code, a traditional software measuring stick. Are more lines of code a good thing, meaning the team has been productive, or a bad thing, meaning the team is writing inefficient spaghetti-style code?

发现的缺陷数量如何?根据测试人员发现的缺陷数量来判断测试人员是否有意义?这如何帮助他们更好地完成工作?可以肯定地说,每行代码产生更多缺陷的开发团队做得不好吗?或者发现更多缺陷的团队做得很好?即使这个想法站得住脚,一个团队被数字打得头破血流又有多大的动力?这会让团队成员开始编写无缺陷的代码吗?

What about number of defects found? Does it make any sense to judge testers by the number of defects they found? How does that help them do their jobs better? Is it safe to say that a development team that produces a higher number of defects per lines of code is doing a bad job? Or that a team that finds more defects is doing a good job? Even if that thought holds up, how motivating is it for a team to be whacked over the head with numbers? Will that make the team members start writing defect-free code?

交流指标

Communicating Metrics

我们知道,无论我们衡量什么,都必然会发生变化。有多少测试正在运行并通过?距离我们需要“现成的构建”还有多少天?完整构建是否通过?我们看不到和容易解释的指标不值得拥有。如果您想跟踪通过测试的数量,请确保该指标以正确的方式对正确的人可见。可见的大图表是我们所知的显示指标的最有效方式。

We know that whatever we measure is bound to change. How many tests are running and passing? How many days until we need a “build on the shelf”? Is the full build passing? Metrics we can’t see and easily interpret aren’t worth having. If you want to track the number of passing tests, make sure that metric is visible in the right way, to the right people. Big visible charts are the most effective way of displaying metrics we know.

丽莎的故事

Lisa’s Story

我以前的团队的目标与单元测试的数量有关。但是,单元测试通过的数量并没有告诉任何人;没有大的可见图表或引用该数字的构建电子邮件。有趣的是,该团队从未在自动化单元测试方面受到关注。

My previous team had goals concerned with the number of unit tests. However, the number of unit tests passing wasn’t communicated to anyone; there were no big visible charts or build emails that referred to that number. Interestingly, the team never got traction on automating unit tests.

在我现在的公司,公司里的每个人都会定期收到单元、GUI 背后和 GUI 级别的通过测试数量的报告(示例请参见表5-15-2 ) 。当这个数字下降而不是上升时,商界人士确实会注意到。随着时间的推移,该团队积累了大量有用的测试。

At my current company, everyone in the company regularly gets a report of the number of passing tests at the unit, behind-the-GUI, and GUI levels (see Tables 5-1 and 5-2 for examples). Business people do notice when that number goes down instead of up. Over time, the team has grown a huge number of useful tests.

——丽莎

—Lisa

表 5-1开始和结束指标

Table 5-1 Starting and Ending Metrics

图像

表 5-2每日构建结果

Table 5-2 Daily Build Results

图像

你的指标值得麻烦吗?不要为了产生数字而测量。想想你将从这些数字中学到什么。在下一节中,我们将考虑您可以从指标中获得的投资回报。

Are your metrics worth the trouble? Don’t measure for the sake of producing numbers. Think about what you’ll learn from those numbers. In the next section, we consider the return on investment you can expect from metrics.

指标投资回报率

Metrics ROI

当您确定所需的指标时,请确保您可以以合理的成本获得它们。如果您的持续构建提供了有用的数字,那么它会带来很好的价值。无论如何,您都在运行构建,如果它给我们提供了额外的信息,那就太好了。如果你需要做很多额外的工作来获取信息,问问自己这是否值得。

When you identify the metrics you need, make sure you can obtain them at a reasonable cost. If your continual build delivers useful numbers, it delivers good value. You’re running the build anyway, and if it gives us extra information, that’s gravy. If you need a lot of extra work to get information, ask yourself if it’s worth the trouble.

Lisa 的团队在跟踪每个故事实际花费的时间与估计时间方面遇到了很多麻烦。除了显而易见的事实,他们还学到了什么那估计只是这样吗?不多。一些有经验的团队发现他们可以省去冲刺燃尽图,因为任务板给了他们足够的信息来衡量他们的进度。他们可以将花费的时间用于估计任务并计算剩余时间以进行更有成效的活动。

Lisa’s team went to a fair amount of trouble to track actual time spent per story versus estimated time. What did they learn other than the obvious fact that estimates are just that? Not much. Some experienced teams find they can dispense with the sprint burndown chart because the task board gives them enough information to gauge their progress. They can use the time spent estimating tasks and calculating the remaining hours on more productive activities.

这并不意味着我们建议您停止跟踪这些测量值。新团队需要了解他们的速度和燃尽率,这样他们才能稳步提高。

This doesn’t mean we recommend that you stop tracking these measurements. New teams need to understand their velocity and burndown rate, so that they can steadily improve.

缺陷率是传统的软件指标,对于以零缺陷为目标的团队而言,它们可能没有太大价值。了解在开发过程中发现和修复错误的比率没有多大价值,因为发现和修复错误是开发过程中不可或缺的一部分。如果测试人员向正在处理代码的程序员展示了一个缺陷,并且编写了单元测试并立即修复了错误,则通常不需要记录缺陷。另一方面,如果许多缺陷在未被发现的情况下进入生产阶段,那么跟踪这些缺陷的数量以了解团队是否有所改进是有价值的。

Defect rates are traditional software metrics, and they might not have much value on a team that’s aiming for zero defects. There’s not much value in knowing the rate of bugs found and fixed during development, because finding and fixing them is an integral part of development. If a tester shows a defect to the programmer who’s working on the code, and a unit test is written and the bug is fixed right away, there’s often no need to log a defect. On the other hand, if many defects reach production undetected, there can be value in tracking the number to know if the team improves.

当它开始重写有缺陷的遗留应用程序时,Lisa 的团队设定了一个目标,即在新代码投入生产后六个月内报告的新代码中的高严重性缺陷不超过六个。拥有一个直接且易于跟踪的目标有助于激励团队找到在开发过程中阻止错误并超越该目标的方法。

When it started to rewrite its buggy legacy application, Lisa’s team set a goal of no more than six high-severity bugs in new code reported after the code is in production over a six-month period. Having a target that was straightforward and easy to track helped motivate the team to find ways to head bugs off during development and exceed this objective.

计算每个指标的投资回报率并决定是跟踪还是维护它。收集它所花费的努力是否证明它提供的价值是合理的?是否易于沟通和理解?一如既往,做适合你情况的事情。尝试在几个冲刺中保持特定的指标,并评估它是否有效。

Figure each metric’s return on investment and decide whether to track or maintain it. Does the effort spent collecting it justify the value it delivers? Can it be easily communicated and understood? As always, do what works for your situation. Experiment with keeping a particular metric for a few sprints and evaluate whether it’s paying off.

与软件质量相关的一种常见指标是缺陷率。在下一节中,我们将探讨跟踪缺陷或不跟踪缺陷的原因,以及我们可以从中学到什么。

One common metric that relates to software quality is the defect rate. In the next section, we look at reasons to track defects, or to not track defects, and what we can learn from them.

缺陷追踪

Defect Tracking

每个新的敏捷团队都会问的一个问题是,“我们是否仍在缺陷跟踪系统中跟踪错误?” 没有简单的答案,但我们会就此事向您提供我们的意见并提供一些替代方案,以便您确定适合您的团队的方案。

One of the questions that are asked by every new agile team is, “Do we still track bugs in a defect tracking system?” There’s no simple answer, but we’ll give you our opinion on the matter and offer some alternatives so that you can determine what fits your team.

为什么我们应该使用缺陷跟踪系统 (DTS)?

Why Should We Use a Defect Tracking System (DTS)?

我们中的许多测试人员都将缺陷跟踪作为交流我们看到的问题的唯一方式,并且很容易继续使用我们熟悉的工具。DTS 是一个方便的地方,不仅可以跟踪缺陷,还可以跟踪优先级、严重性和状态,并查看它被分配给了谁。许多敏捷实践者说我们不需要再这样做了,我们可以通过卡片或其他一些简单的机制来跟踪缺陷。我们可以编写一个测试来显示失败,修复代码,并将测试保留在我们的回归套件中。

A lot of us testers have used defect tracking as the only way to communicate the issues we saw, and it’s easy to keep using the tools we are familiar with. A DTS is a convenient place to keep track of not only the defect but the priorities, severities, and status, and to see who it is assigned to. Many agile practitioners say that we don’t need to do this anymore, that we can track defects on cards or some other simple mechanism. We could write a test to show the failure, fix the code, and keep the test in our regression suite.

但是,有理由继续使用工具来记录缺陷及其修复方式。现在让我们探讨其中的一些。

However, there are reasons to keep using a tool to record defects and how they were fixed. Let’s explore some of them now.

方便

不保留缺陷跟踪系统的担忧之一是没有地方保留错误的所有细节。测试人员习惯于记录带有大量信息的错误,例如如何重现它,它是在什么环境中发现的,或者使用的是什么操作系统或浏览器。所有这些信息都无法放在一张卡片上,那么您如何捕捉这些细节呢?如果你只依赖卡片,你还需要交谈。但是通过谈话,细节会丢失,有时测试人员会忘记到底做了什么——特别是如果在程序员解决问题前几天发现了错误。

One of the concerns about not keeping a defect tracking system is that there is no place to keep all of the details of the bug. Testers are used to recording a bug with lots of information, such as how to reproduce it, what environment it was found in, or what operating system or browser was used. All of this information cannot fit on a card, so how do you capture those details? If you are relying only on cards, you also need conversation. But with conversation, details get lost, and sometimes a tester forgets exactly what was done—especially if the bug was found a few days prior to the programmer tackling the issue.

DTS 也是保存所有补充文档(如屏幕打印或上传文件)的便利位置。

A DTS is also a convenient place to keep all supplemental documentation, such as screen prints or uploaded files.

知识库

我们听说过跟踪缺陷的理由,例如“我们需要能够查看旧的错误报告”。我们试图思考为什么您需要查看旧错误报告的原因,并且在我们研究本章时,珍妮特找到了一个例子。

We have heard reasons to track defects such as, “We need to be able to look at old bug reports.” We tried to think of reasons why you would ever need to look at old bug reports, and as we were working on this chapter, Janet found an example.

珍妮特的故事

Janet’s Story

当我在西捷测试预选算法时,我发现了一个异常。我问另一位测试员桑德拉,她以前是否遇到过这个问题。桑德拉依稀记得一些事情,但不完全是当时的情况。她很快在 Bugzilla 中进行了搜索,并立即找到了问题所在。它因无效而被关闭,因为企业认为不值得花时间修复它,而且影响很小。

When I was testing the pre-seating algorithm at WestJet, I found an anomaly. I asked Sandra, another tester, if she had ever come across the issue before. Sandra vaguely recalled something about it but not exactly what the circumstances were. She quickly did a search in Bugzilla and found the issue right away. It had been closed as invalid because the business had decided that it wasn’t worth the time it would take to fix it, and the impact was low.

能够查找它使我免于四处奔波试图提出问题或重新输入错误并再次关闭它。因为团队成员坐彼此靠近,我们的谈话导致了与团队中业务分析师的另一次谈话。这次谈话激发了创建常见问题解答页面、未决问题列表或类似内容的想法,这些内容将为新测试人员提供一个位置来查找所有已确定但已决定不解决的问题。

Being able to look it up saved me from running around trying to ask questions or reentering the bug and getting it closed again. Because the team members sit close to each other, our talking led to another conversation with the business analyst on the team. This conversation sparked the idea of a FAQ page, an outstanding issues list, or something along that line that would provide new testers a place to find all of the issues that had been identified but for which the decision had been made not to address them.

——珍妮特

—Janet

这个故事表明,虽然错误数据库可以用作知识库,但可能还有其他机制来保存业务决策及其背景信息。如果一个 issue 太老以至于已经忘记了,也许我们应该重写它并再次提出来。情况可能已经改变,企业可能会决定现在修复错误是值得的。

This story shows that although the bug database can be used as a knowledge base, there might be other mechanisms for keeping business decisions and their background information. If an issue is old enough to have been lost track of, maybe we should rewrite it and bring it up again. The circumstances may have changed, and the business might decide it is now worthwhile to fix the bug.

便于保留在 DTS 中的错误类型是间歇性的并且需要很长时间才能追踪到的错误。这些错误很少出现,并且通常会出现时间间隔,在此期间调查会因缺乏信息而停滞不前。DTS 是一个可以捕获有关到目前为止已计算出的信息的地方。它还可以包含日志、跟踪等。当团队中的某个人终于有时间查看问题或问题变得更加关键时,这可能是有价值的信息。

The types of bugs that are handy to keep in a DTS are the ones that are intermittent and take a long time to track down. These bugs present themselves infrequently, and there are usually gaps in time during which the investigation stalls for lack of information. A DTS is a place where information can be captured about what was figured out so far. It can also contain logs, traces, and so on. This can be valuable information when someone on the team finally has time to look at the problem or the issue becomes more critical.

错误报告中的信息以后可以用于多种目的。这是 Lisa 团队关于如何使用其信息的故事。

The information in bug reports can be used later for several purposes. Here’s a story from Lisa’s team on how it uses its information.

丽莎的故事

Lisa’s Story

我们团队的一名开发人员在每次迭代中轮流担任“生产支持”。生产支持请求来自业务方,要求手动修复过去的错误或需要人工干预的生产问题。“生产支持人员”研究问题并在错误报告中记录为修复它所做的一切。这些注释通常包括 SQL 语句和有关原因的信息。如果以后有人遇到同样的错误或情况,可以很容易地在 DTS 中找到解决方案。如果某些类型的问题似乎经常出现,团队可以使用 DTS 进行研究和分析。尽管我们的团队很小,但我们要处理很多遗留代码,我们不能依靠人们的记忆来跟踪每个问题并进行修复。

One developer from our team serves on a “production support” rotation for each iteration. Production support requests come in from the business side for manual fixes of past mistakes or production problems that need manual intervention. The “production support person” researches the problem and notes whatever was done to fix it in the bug report. These notes usually include a SQL statement and information about the cause. If anyone encounters the same error or situation later, the solution can be easily found in the DTS. If certain types of problems seem to occur frequently, the team can use the DTS for research and analysis. Even though our team is small, we deal with a lot of legacy code, and we can’t rely on people’s memory to keep track of every problem and fix.

——丽莎

—Lisa

当团队特别大或不在同一地点时,记住缺陷的原因或为满足特殊要求所做的工作就更难了。客户也可能对其问题的解决方案感兴趣。

Remembering the cause of defects or what was done to fulfill a special request is even harder when the team is particularly large or isn’t co-located. Customers might also be interested in the solutions to their problems.

大型或分布式团队

如果项目太大以至于一个团队发现的缺陷可能会影响其他团队,那么 DTS 可能是一个不错的选择。当然,要发挥作用,团队的所有成员都需要能够访问它。面对面交流始终是我们的首选,但当情况变得不切实际时,我们需要 DTS 等辅助工具。

If projects are so large that defects found by one team might affect other teams, a DTS is probably a good choice. Of course, to be useful it needs to be accessible to all members of the team. Face-to-face communication is always our first choice, but when circumstances make that impractical, we need aids such as a DTS.

客户支持

当发布后有客户报告的缺陷时,客户通常想知道这些缺陷何时得到修复。对于帮助台或技术支持来说,了解给定版本中修复了什么是非常宝贵的。他们还可以找到发布时仍未解决的缺陷,并告知客户。DTS 使将这些信息集中在一起变得更加简单。

When there are defects that have been reported by the customer after the release, the customer usually wants to know when they’ve been fixed. It’s invaluable for the help desk or technical support to know what was fixed in a given release. They can also find defects that are still outstanding at release time and let the customers know. A DTS makes it much simpler to pull this information together.

指标

跟踪缺陷率是有原因的。也有一些您不跟踪缺陷的原因。例如,我们认为如果 bug 从未脱离迭代,则不应将其视为缺陷。当然,这会引发另一个关于我们应该跟踪什么以及为什么要跟踪的讨论,但我们不会在这里讨论。

There are reasons to track defect rates. There are also reasons why you wouldn’t track a defect. For example, we don’t think that a bug should be counted as a defect if it never makes it out of the iteration. This, of course, brings up another discussion about what should we track and why, but we won’t discuss that here.

第 18 章编码和测试”探讨了与缺陷率相关的指标。

Chapter 18, “Coding and Testing,” explores metrics related to defect rates.

可追溯性

我们听说拥有 DTS 的另一个原因是可追溯性,将缺陷链接到测试用例。我们不确定这是一个正当理由。并非所有缺陷都与测试用例有关,也不应该与测试用例有关。例如,拼写错误之类的错误可能不需要特定的测试用例。也许该产品使用起来不直观;这是一个非常真实的错误,通常不会被报告。您如何编写测试以确定某些东西是否可用?探索性测试可能会发现边缘条件中的错误,这些错误不值得创建自动化测试。

Another reason we’ve heard for having a DTS is traceability, linking defects to test cases. We’re not sure that this is a valid reason. Not all defects are linked to test cases, nor should they be. For example, errors like spelling mistakes might not need specific test cases. Maybe the product was not intuitive to use; this is a very real bug that often goes unreported. How do you write a test to determine if something is usable? Exploratory testing might find bugs in edge conditions that are not worth the effort of creating automated tests.

如果它是一个自动测试用例捕获了一个错误,那么记录该缺陷的需要会进一步减少,因为如果重新引入它会再次被捕获。对可追溯性的需求消失了。所以,也许我们不需要跟踪缺陷。

If it is an automated test case that caught a bug, then the need to record that defect is further reduced, because it will be caught again if ever reintroduced. The need for traceability is gone. So, maybe we don’t need to track defects.

为什么我们不应该使用 DTS?

Why Shouldn’t We Use a DTS?

敏捷和精益为我们提供了有助于减少对 DTS 需求的实践和原则。如果过程是可靠的,并且所有人都致力于提供高质量的产品,那么缺陷应该很少见并且很容易被跟踪。

Agile and Lean provide us with practices and principles that help reduce the need for a DTS. If the process is solid, and all of the people are committed to delivering a quality product, defects should be rare and very simply tracked.

作为交流工具

缺陷跟踪系统当然不会促进程序员和测试人员之间的交流。他们可以很容易地避免直接与对方交谈。

Defect tracking systems certainly don’t promote communication between programmers and testers. They can make it easy to avoid talking directly to each other.

浪费时间和库存

除了重现缺陷的所有步骤之外,我们倾向于将大量信息放入 DTS。根据错误的不同,可能需要很长时间才能编写这些步骤,以便程序员也可以重现它。然后是分类,必须有人发表评论、解释缺陷、尝试重现它、(理想情况下)修复它、写更多评论,然后将它分配回报告它的人。最后,可以验证修复。如果程序员一开始就误解了问题,整个周期可能会加倍。单个缺陷报告的成本可能会变得过高。

We tend to put lots of information into the DTS in addition to all of the steps to reproduce the defect. Depending on the bug, it can take a long time to write these steps so that the programmer can reproduce it as well. Then there is the triage, and someone has to make comments, interpret the defect, attempt to reproduce it, (ideally) fix it, write more comments, and assign it back to the person who reported it. Finally, the fix can be verified. This whole cycle can double if the programmer misunderstood the problem in the first place. The cost of a single defect report can become exorbitant.

如果我们作为测试人员可以与程序员交谈并展示我们发现的内容,然后开发人员立即修复缺陷,那会容易得多?我们稍后会详细讨论。

How much easier would it be if we as testers could just talk to the programmer and show what we found, with the developer then fixing the defect right away? We’ll talk more about that later.

第 15 章编码和测试”中,我们将解释测试人员和程序员如何协同工作来解决错误。

In Chapter 15, “Coding and Testing,” we’ll explain how tester and programmers work together on bugs.

DTS 中的缺陷成为队列或迷你产品积压。根据精益原则,这种缺陷清单是一种浪费。作为一个团队,我们应该想办法减少这种浪费。

Defects in a DTS become a queue or a mini product backlog. According to lean principles, this inventory of defects is a waste. As a team, we should be thinking of ways to reduce this waste.

珍妮特的故事

Janet’s Story

2004 年, TestingReflections.com的作者 Antony Marcano写了一篇关于不使用错误跟踪系统的想法的博客文章。当它在邮件列表中被讨论时,他被许多测试者抨击为引入了类似于异端邪说的东西。他发现他现在有了不同的接受方式,因为这个想法正在进入敏捷思维的主流。

In 2004, Antony Marcano, author of TestingReflections.com, wrote a blog post about the idea of not using a bug-tracking system. When it was discussed on mailing lists, he was flamed by many testers as introducing something similar to heresy. He finds he has a different reception now, because the idea is making its way into the mainstream of agile thinking.

他建议敏捷团队中的错误跟踪系统只是“秘密积压”。

He suggests that bug-tracking systems in agile teams are just “secret backlogs.”

——珍妮特

—Janet

当我们在第 18 章编码和测试”中介绍迭代计划时,Antony 将分享他关于隐藏积压的想法。

Antony will share his ideas about the hidden backlog when we cover iteration planning in Chapter 18, “Coding and Testing.”

缺陷跟踪工具

Defect Tracking Tools

如果您决定使用 DTS,请谨慎选择。了解您的需求并保持简单。您会希望团队中的每个人都使用它。如果它变得开销很大或难以使用,人们会想方设法解决它。对于敏捷开发团队使用的所有工具,您应该考虑整个团队的意见。如果客户团队的任何人输入错误报告,也请听取他或她的意见。

If you do decide to use a DTS, choose it carefully. Understand your needs and keep it simple. You will want everyone on the team to use it. If it becomes overhead or hard to use, people will find ways to work around it. As with all tools used by your agile development team, you should consider the whole team’s opinion. If anyone from the customer team enters bug reports, get his or her opinion too.

Janet 使用过的最简单的工具之一是 Alcea 的 FIT IssueTrack。它是可配置的,不会让您遵循预定义的流程,并且很容易从中获取指标。做好功课,找到适合您的工具。有多种开源缺陷跟踪系统、托管系统和集成企业系统可用。

One of the simplest tools that Janet has used is Alcea’s FIT IssueTrack. It is configurable, does not make you follow a predefined process, and is easy to get metrics out of. Do your homework and find the tool that works for you. There are a variety of open source defect-tracking systems, hosted systems, and integrated enterprise systems available.

无论您是否使用 DTS,您都希望缺陷尽可能可见。

Whether or not you use a DTS, you want to make defects as visible as possible.

我们使用商业 DTS,但我们发现保持错误可见的价值。我们对错误进行颜色编码,并将它们作为任务包含在我们的故事板中,如图5-1所示。黄牌表示正常错误,红牌表示高生产错误或“测试停止”开发错误——这两类都需要立即解决。快速查看看板可以让我们看到待办事项、WIP、验证和完成列中有多少错误。其他卡片也采用颜色编码:故事卡片为蓝色,测试任务卡片为绿色,开发任务卡片为白色。条纹卡片用于迭代计划后添加的任务。黄色和红色的虫子卡很容易脱颖而出。

We use a commercial DTS, but we find value in keeping bugs visible. We color-code bugs and include them as tasks in our story board, shown in Figure 5-1. Yellow cards denote normal bugs, and red cards denote either high production bugs or “test stopper” development bugs—both categories need to be addressed right away. A quick look at the board lets us see how many bugs are in the To Do, WIP, Verify and Done columns. Other cards are color-coded as well: blue for story cards, green for test task cards, and white for development tasks. Striped cards are for tasks added after iteration planning. Yellow and red bug cards stand out easily.

图 5-1带有颜色编码卡片的故事板。经 Mike Thomas 许可使用。版权所有 2008。

Figure 5-1 Story board with color-coded cards. Used with permission of Mike Thomas. Copyright 2008.

图像

在我们写这本书的时候,我的团队转换为虚拟故事板,因为我们的一个团队成员成为了远程团队成员,但我们保留了这种颜色编码的概念。

During the time we were writing this book, my team converted to a virtual story board because one of our team members became a remote team member, but we retained this color-coding concept.

——丽莎

—Lisa

我们通常建议尝试使用不同的工具,使用每一种工具进行几次迭代,但这对于错误跟踪系统来说比较棘手,因为您需要将一个系统中的所有错误移植到您正在尝试的新系统中尺寸。花一些时间思考您在 DTS 中需要什么,它将服务于什么目的,并明智地评估备选方案。

We usually recommend experimenting with different tools, using each one for a few iterations, but this is trickier with bug-tracking systems, because you need to port all of the bugs that are in one system to the new one that you’re trying on for size. Spend some time thinking about what you need in a DTS, what purposes it will serve, and evaluate alternatives judiciously.

丽莎的故事

Lisa’s Story

我的团队使用的是基于 Web 的 DTS,这基本上是由管理层强加给它的。我们发现它使用起来有些麻烦,缺乏基本功能,例如错误报告的时间戳更新,我们对许可限制感到恼火。我们的测试人员对我们的许可证限制我们只能同时使用三个用户这一事实感到特别沮丧,因此会话设置为很快超时。

My team used a web-based DTS that was basically imposed upon it by management. We found it somewhat cumbersome to use, lacking in basic features such as time-stamping updates to the bug reports, and we chafed at the license restrictions. We testers were especially frustrated by the fact that our license limited us to three concurrent users, so sessions were set to time out quickly.

该团队留出时间来评估不同的 DTS 备选方案。起初,选择似乎令人难以置信。但是,我们找不到一种工具可以满足我们的所有要求。每个工具似乎都缺少一些重要的东西,或者我们从使用过该工具的人那里听到了负面报告。我们担心将现有错误数据库转换为新系统所需的工作量。

The team set aside time to evaluate different DTS alternatives. At first, the selection seemed mind-boggling. However, we couldn’t find one tool that met all our requirements. Every tool seemed to be missing something important, or we heard negative reports from people who had used the tool. We were concerned about the effort needed to convert the existing bug database into a new system.

当我们的 DTS 实际崩溃时,这个问题是被迫的。几年前我们已经停止支付支持费用,但系统管理员决定查看供应商对该工具进行了哪些改进。他发现我们遇到的很多缺点都得到了解决。例如,所有更新现在都带有时间戳。一个客户端应用程序可用,它不受会话超时的影响,并且具有对测试人员特别有价值的增强功能。

The issue was forced when our DTS actually crashed. We had stopped paying for support a couple of years earlier, but the system administrator decided to see what enhancements the vendor had made in the tool. He found that a lot of shortcomings we had experienced had been addressed. For example, all updates were now time stamped. A client application was available that wasn’t subject to session timeouts and had enhanced features that were particularly valuable to the testers.

通过使用我们现有的工具并支付升级和维护费用,加上允许更多并发用户的许可证,我们在将现有数据转换为新版本方面获得了帮助,并以低成本轻松获得了一个工作系统。一个好处是我们的客户不需要学习新系统。

By going with our existing tool and paying for the upgrade and maintenance, plus a license allowing more concurrent users, we got help with converting our existing data to the new version and got a working system easily and at a low cost. A bonus was that our customers weren’t faced with having to learn a new system.

有时最好的工具就是您已经拥有的工具,如果您只是看看它是如何改进的!

Sometimes the best tool is the one you already have if you just look to see how it has improved!

——丽莎

—Lisa

与所有工具搜索一样,向社区中的其他人(例如用户组和邮件列表)寻求建议。在开始寻找之前定义您的标准,并尽可能多地进行实验。如果您选择了错误的工具,请减少损失并开始研究替代品。

As with all your tool searches, look to others in your community, such as user groups and mailing lists, for recommendations. Define your criteria before you start looking, and experiment as much as you can. If you choose the wrong tool, cut your losses and start researching alternatives.

保持专注

Keep Your Focus

关于报告和跟踪缺陷的决定很重要,但不要忘记您的主要目标。您希望尽可能提供最优质的产品,并且希望及时为企业创造价值。项目当允许人们尽力而为时,就会成功。专注于改善沟通和建立协作。如果您遇到很多缺陷,请调查问题的根源。如果您需要 DTS 来执行此操作,请使用它。如果您的团队通过记录可执行测试中的缺陷并立即修复它们来更好地工作,那就去做吧。如果某种组合能让你不断改进,那就去吧。要记住的主要事情是它必须适用于您的整个团队。

Decisions about reporting and tracking defects are important, but don’t lose track of your main target. You want to deliver the best quality product you can, and you want to deliver value to the business in a timely manner. Projects succeed when people are allowed to do their best work. Concentrate on improving communication and building collaboration. If you encounter a lot of defects, investigate the source of the problem. If you need a DTS to do that, use it. If your team works better by documenting defects in executable tests and fixing them right away, do that. If some combination enables you continually improve, go with it. The main thing to remember is that it has to work for your whole team.

第 18 章,“编码和测试”涵盖了备选方案,并向您展示了解决错误问题的不同方法。

Chapter 18, “Coding and Testing,” covers alternatives and shows you different ways to attack your bug problems.

缺陷跟踪是敏捷测试中产生最多问题和争议的典型质量流程之一。混淆的另一个重要来源是敏捷项目是否需要测试计划或可追溯性矩阵等文档。接下来让我们考虑一下。

Defect tracking is one of the typical quality processes that generate the most questions and controversy in agile testing. Another big source of confusion is whether agile projects need documents such as test plans or traceability matrices. Let’s consider that next.

测试计划

Test Planning

传统的分阶段软件方法强调测试计划作为整体文档需求的一部分的重要性。它们旨在为涉众概述软件测试工作的目标、范围、方法和重点。完成的文档旨在帮助测试组以外的人员了解产品验证的“原因”和“方式”。在本节中,我们将研究测试计划以及准备和跟踪敏捷项目测试工作的其他方面。

Traditional phased software methodologies stress the importance of test plans as part of the overall documentation needs. They’re intended to outline the objectives, scope, approach, and focus of the software testing effort for stakeholders. The completed document is intended to help people outside the test group understand the “why” and “how” of product validation. In this section, we look at test plans and other aspects of preparing and tracking the testing effort for an agile project.

测试策略与测试计划

Test Strategy vs. Test Planning

在敏捷项目中,团队不依赖大量文档来传达测试人员需要做什么。测试人员与团队的其他成员携手合作,以便测试工作以任务卡的形式对所有人可见。所以经常向我们提出的问题是,“是否还需要测试计划?” 要回答这个问题,让我们首先看一下测试计划和测试策略或方法之间的区别。

In an agile project, teams don’t rely on heavy documentation to communicate what the testers need to do. Testers work hand in hand with the rest of the team so that the testing efforts are visible to all in the form of task cards. So the question often put to us is, “Is there still a need for test plans?” To answer that question, let’s first take a look at the difference between a test plan and a test strategy or approach.

文档中包含的信息越多,有人阅读所有内容的可能性就越小。考虑利益相关者真正需要哪些信息。想想它的使用频率和用途。

The more information that is contained in a document, the less likely it is that someone is going to read it all. Consider what information is really necessary for the stakeholders. Think about how often it is used and what it is used for.

我们喜欢将测试策略视为很少更改的静态文档,而测试计划是新创建的并且特定于每个新项目。

We like to think of a test strategy as a static document that seldom changes, while a test plan is created new and is specific to each new project.

测试策略

战略是一项长期的行动计划,关键词是“长期”。如果您的组织需要有关您的整体测试方法的文档项目,请考虑获取此信息并将其放入静态文档中,该文档不会随时间发生太大变化。有很多信息不是项目特定的,可以提取到测试策略或测试方法文档中。

A strategy is a long-term plan of action, the key word being “long-term.” If your organization wants documentation about your overall test approach to projects, consider taking this information and putting it in a static document that doesn’t change much over time. There is a lot of information that is not project specific and can be extracted into a Test Strategy or Test Approach document.

然后可以将此文档用作参考,并且只有在流程发生变化时才需要更新。测试策略文档可用于让新员工对您的测试过程如何工作有一个更高层次的理解。

This document can then be used as a reference and needs to be updated only if processes change. A test strategy document can be used to give new employees a high-level understanding of how your test processes work.

珍妮特的故事

Janet’s Story

我在几个组织中使用这种方法取得了成功。所有项目共有的流程被收集到一份文件中。使用此格式可以满足大多数合规性要求。所涵盖的一些主题是:

I have had success with this approach at several organizations. Processes that were common to all projects were captured into one document. Using this format answered most compliance requirements. Some of the topics that were covered were:

• 测试实践

• Testing Practices

• 故事测试

• Story Testing

• 解决方案验证测试

• Solution Verification Testing

• 用户验收测试

• User Acceptance Testing

• 探索性测试

• Exploratory Testing

• 负载和性能测试

• Load and Performance Testing

• 测试自动化

• Test Automation

• 检测结果

• Test Results

• 缺陷跟踪过程

• Defect Tracking Process

• 测试工具

• Test Tools

• 测试环境

• Test Environments

——珍妮特

—Janet

测试计划

规划的力量在于识别可能的问题和依赖性,将风险浮出水面以供讨论和解决,并考虑大局。测试计划也不例外。团队应该在开始之前考虑每个项目的风险和依赖关系以及全局。

The power of planning is to identify possible issues and dependencies, to bring risks to the surface to be talked about and to be addressed, and to think about the big picture. Test planning is no different. A team should think about risks and dependencies and the big picture for each project before it starts.

无论您的团队是否决定创建测试计划文档,都应该进行计划。每个项目都是不同的,所以不要指望相同的解决方案适用于所有项目。

Whether your team decides to create a test plan document or not, the planning should be done. Each project is different, so don’t expect that the same solution will fit all.

第 15 章发布或主题规划中的测试人员活动”中,我们展示了示例并讨论了您在规划发布时可以使用的替代方案。

In Chapter 15, “Tester Activities in Release or Theme Planning,” we show examples and discuss alternatives you can use when you are planning the release.

有时我们的客户坚持要一份测试计划文件。如果您签订合同开发应用程序,则测试计划可能是一组可交付成果的一部分,这些交付成果还包括需求文档和设计文档等项目。

Sometimes our customers insist on a test plan document. If you’re contracting to develop an application, a test plan might be part of a set of deliverables that also include items such as a requirements document and a design document.

谈论测试计划通常会导致谈论可追溯性。是否有人对交付的代码执行了预期行为的所有计划测试?需求和测试计划如何与实际测试和最终功能相​​关联?

Talk of test plans often leads to talk of traceability. Did someone execute all planned testing of the desired behavior on the delivered code? How do requirements and test plans relate to the actual testing and final functionality?

可追溯性

Traceability

在传统项目中,我们过去常常需要可追溯性矩阵来确定我们是否实际测试了所有需求。如果需求发生变化,我们需要知道我们已经更改了适当的测试用例。对于非常大的需求文档,这是测试团队知道它具有良好覆盖率的唯一方式。

In traditional projects, we used to need traceability matrices to determine whether we had actually tested all of the requirements. If a requirement changed, we needed to know that we had changed the appropriate test cases. With very large requirements documents, this was the only way that a test team knew it had good coverage.

在敏捷项目中,我们没有这些限制。我们以微小的、定义明确的步骤构建功能。我们与团队密切合作,知道什么时候会发生变化。如果程序员先进行测试,我们就会知道所有小块工作都有单元测试。然后我们可以与客户合作定义验收测试。我们在程序员处理每个故事时对其进行测试,因此我们知道没有什么是未经测试的。

In an agile project, we don’t have those restrictions. We build functionality in tiny, well-defined steps. We work with the team closely and know when something changes. If the programmers work test-first, we know there are unit tests for all of the small chunks of work. We can then collaborate with the customer to define acceptance tests. We test each story as the programmer works on it, so we know that nothing goes untested.

受监管行业可能需要某种可追溯性。如果有,我们建议你真正看看问题管理试图解决什么问题。当您了解需要什么时,您应该尝试使解决方案尽可能简单。有多种方法可以提供可追溯性。源代码签入评论可以参考包含需求或测试用例的 wiki 页面,或者参考缺陷号。您可以在单元测试中添加注释,将测试与需求的位置或标识符联系起来。测试可以直接与 FitNesse 等工具中的需求集成。您的团队可以轻松找到最适合客户需求的方式。

There might be requirements for some kind of traceability for regulated industries. If there is, we suggest that you really look at what problem management is trying to solve. When you understand what is needed, you should try to make the solution as simple as possible. There are multiple ways to provide traceability. Source code check-in comments can refer to the wiki page containing the requirements or test cases, or to a defect number. You can put comments in unit tests tying the test to the location or identifier of the requirement. The tests can be integrated directly with the requirements in a tool such as FitNesse. Your team can easily find the way that works best for your customers’ needs.

可能需要诸如可追溯性矩阵之类的文件来满足组织的审计标准或质量模型强加的要求。让我们考虑一下这些指令如何与敏捷开发相得益彰。

Documents such as traceability matrices might be needed to fulfill requirements imposed by the organization’s audit standards or quality models. Let’s consider how these directives get along with agile development.

现有流程和模型

Existing Processes and Models

这个问题经常被问到:“传统的质量模型和流程能否与敏捷开发方法共存?” 从理论上讲,他们没有理由不能。实际上,往往别无选择。质量模型通常属于传统 QA 团队的领域,他们也可以跟随测试人员进入新的敏捷结构。将它们融入新的敏捷开发模型可能并不容易。让我们看看一些典型的质量流程,以及测试人员和他们的团队如何适应这些流程。

This question is often asked: “Can traditional quality models and processes coexist with agile development methods?” In theory, there is no reason why they can’t. In reality, there is often not a choice. Quality models often fall into the domain of the traditional QA team, and they can follow testers into the new agile structure as well. It might not be easy to fit these into a new agile development model. Let’s look at a few typical quality processes and how testers and their teams might accommodate them.

审计

Audits

不同的行业有不同的审计要求。传统开发组织中的质量保证团队通常负责为审计员提供信息并确保符合审计要求。为应对引人注目的公司财务丑闻而颁布的 2002 年萨班斯-奥克斯利法案规定了维护业务记录的要求。确保合规性通常落在 IT 部门的肩上。SAS 70 是另一个广泛认可的服务组织审计标准。这些只是影响开发团队的审计控制类型的几个示例。

Different industries have different audit requirements. Quality assurance teams in traditional development organizations are often tasked with providing information for auditors and ensuring compliance with audit requirements. The Sarbanes-Oxley Act of 2002, enacted in response to high-profile corporate financial scandals, sets out requirements for maintaining business records. Ensuring compliance usually falls to the IT departments. SAS 70 is another widely recognized auditing standard for service organizations. These are just a couple of examples of the type of audit controls that affect development teams.

较大的组织有专门的团队来控制合规性并与审计员合作,但开发团队经常被要求提供信息。示例包括在给定的软件版本上执行了哪些测试,或证明不同的帐户协调一致。测试人员的任务是编写测试计划来评估控制活动的有效性。

Larger organizations have specialized teams that control compliance and work with auditors, but development teams are often asked to provide information. Examples include what testing has been performed on a given software release, or proving that different accounts reconcile. Testers can be tasked with writing test plans to evaluate the effectiveness of control activities.

丽莎的故事

Lisa’s Story

我们公司定期接受 SAS 70 审计。每当安排一个时,我们都会写一张故事卡,为审计提供支持。大部分工作落在系统管理员身上,但我为与审计员一起工作的业务人员提供支持。有时我们需要在演示环境中演示系统功能。如果出现问题,我可以为演示提供数据并提供帮助。我可能还会被要求提供有关我们如何测试特定功能的详细信息。

Our company undergoes regular SAS 70 audits. Whenever one is scheduled, we write a story card for providing support for the audit. Most of this work falls to the system administrators, but I provide support to the business people who work with the auditor. Sometimes we’re required to demonstrate system functionality in our demo environment. I can provide data for the demos and help if questions arise. I might also be asked to provide details about how we tested a particular piece of functionality.

我们的一些内部流程需要符合 SAS 70 要求。例如,每次我们发布到生产环境时,我们都会填写一张表格,其中包含有关发布了哪个版本、每个级别对其运行了多少测试、发布者和验证者的信息。

Some of our internal processes are required to conform with SAS 70 requirements. For example, every time we release to production, we fill out a form with information about which build was released, how many tests at each level were run on it, who did the release, and who verified it.

——丽莎

—Lisa

属于敏捷团队的测试人员应该致力于该团队。如果需要他们的帮助来提供审计信息或帮助确保合规性,为此写故事,并与团队的其他工作一起计划。与合规和内部审计团队合作,了解您团队的职责。

Testers who are part of an agile team should be dedicated to that team. If their help is needed in providing information for an audit or helping to ensure compliance, write stories for this and plan them along with the rest of the team’s work. Work together with the compliance and internal audit teams to understand your team’s responsibilities.

框架、模型和标准

Frameworks, Models, and Standards

质量模型有很多,但我们只看其中两个,以展示如何调整敏捷过程以适应它们的约束。

There are many quality models, but we’ll look at two to show how you can adapt your agile process to fit within their constraints.

1.能力成熟度模型集成 (CMMI) 旨在帮助组织改进他们的过程,但并不规定具体的开发实践来完成改进。

1. The Capability Maturity Model Integration (CMMI) aims to help organizations improve their process but doesn’t dictate specific development practices to accomplish the improvements.

2.信息技术基础架构库 (ITIL) 是一组 IT 服务管理最佳实践,旨在帮助组织开发有效的质量流程。

2. Information Technology Infrastructure Library (ITIL) is a set of best practices for IT service management intended to help organizations develop an effective quality process.

这两种模型都可以与敏捷开发愉快地共存。他们植根于同一个目标,即使软件开发项目取得成功。

Both of these models can coexist happily with agile development. They’re rooted in the same goal, making software development projects succeed.

让我们看看 CMMI,这是一个用于衡量流程成熟度的框架。它通过测量过程是未知的、已定义的、记录的、永久的还是优化的来定义每个级别。敏捷项目有一个明确的过程,尽管并非所有团队都记录他们所做的事情。例如,通过发布计划墙上的索引卡管理您的需求,并由单个客户做出最终决定,只要您一直这样做,就是一个明确的过程。

Let’s look at CMMI, a framework for measuring the maturity of your process. It defines each level by measuring whether the process is unknown, defined, documented, permanent, or optimized. Agile projects have a defined process, although not all teams document what they do. For example, managing your requirements with index cards on a release planning wall with a single customer making the final decisions is a defined process as long as you do it all the time.

回顾旨在不断改进流程,团队应该始终寻找优化流程的方法。如果您的团队唯一缺乏的是文档,那么请考虑将您的过程包含到您的测试策略文档中。

Retrospectives are aimed at constant process improvement, and teams should be always be looking for ways to optimize processes. If the only thing your team is lacking is documentation, then think about including your process into your test strategy documentation.

问问自己,满足 CMMI 要求的最少文档数量是多少。Janet 在使用图 5-2中的图表方面取得了成功。

Ask yourself what the minimum amount of documentation you could give to satisfy the CMMI requirements would be. Janet has had success with using diagrams like the one in Figure 5-2.

图 5-2记录测试策略

Figure 5-2 Documenting the test strategy

图像

有关 CMMI 和敏捷开发的信息,请参阅参考书目。

See the bibliography for information about CMMI and agile development.

如果 ITIL 已被引入您的组织并影响变更管理,请调整您的流程以适应它。您甚至可能会发现新流程是有益的。

If ITIL has been introduced into your organization and affects change management, adapt your process to accommodate it. You might even find the new process beneficial.

珍妮特的故事

Janet’s Story

当我在一个拥有中央呼叫中心处理所有客户支持电话的组织工作时,管理层为组织的服务部分实施了 ITIL。在变更管理团队意识到未解决问题的数量在稳步增加之前,我们认为这不会影响开发团队。没有人理解为什么这个数字一直在上升,所以我们举行了一系列的问题解决会议。首先,我们制定了当前有效的流程。

When I worked in one organization that had a central call center to handle all of the customers’ support calls, management implemented ITIL for the service part of the organization. We didn’t think it would affect the development team until the change management team realized that the number of open problems was steadily increasing. No one understood why the number kept going up, so we held a series of problem-solving sessions. First, we mapped out the process currently in effect.

呼叫中心工作人员在他们的跟踪系统中报告了一起事件。他们试图立即解决客户的问题。通常,这意味着为软件缺陷提供解决方法。呼叫中心报告已关闭,但出现问题Remedy 中的报告随后被打开,开发团队中的某个人收到了一封电子邮件。如果开发团队接受了缺陷,则将缺陷输入 Bugzilla 进行修复。

The call center staff reported an incident in their tracking system. They tried to solve the customer’s problem immediately. Often, that meant providing a work-around for a software defect. The call center report was closed, but a problem report in Remedy was then opened, and someone in the development team was sent an email. If the defect was accepted by the development team, a defect was entered into Bugzilla to be fixed.

当缺陷最终被修复时,没有循环回到问题 issue 来关闭它。我们与所有相关利益相关者举行了几次头脑风暴会议,以确定该问题的最佳和最简单的解决方案。

There was no loop back to the problem issue to close it when the defect was finally fixed. We held several brainstorming sessions with all involved stakeholders to determine the best and easiest solution to that problem.

要解决的问题陈述是,“项目团队如何报告问题并改变管理人员以告诉他们错误何时真正修复?”

The problem statement to solve was, “How does the project team report back to the problem and change management folks to tell them when the bug was actually fixed?”

我们有几种方法可以解决这个问题。一种选择是在 Bugzilla 中引用 Remedy 票并将挂钩放入 Remedy,这样当我们关闭 Bugzilla 缺陷时,Remedy 会检测到它并关闭 Remedy 票。当然,有些错误从未得到解决,这意味着 Remedy 门票永远保持开放状态。

There were a couple of ways we could have solved the problem. One option was to reference the Remedy ticket in Bugzilla and put hooks into Remedy so that when we closed the Bugzilla defect, Remedy would detect it and close the Remedy ticket. Of course, some of the bugs were never addressed, which meant the Remedy tickets stayed open forever.

我们实际上为整个团队找到了更好的解决方案,包括问题变更人员。我们集思广益了很多不同的想法,但决定当 Bugzilla 中出现错误时,我们可以关闭 Remedy 票,因为我们实际上永远不会回到最初的投诉并告诉报告它的客户,或者修复完成的时间.

We actually found a better solution for the whole team, including the problem change folks. We brainstormed a lot of different ideas but decided that when a bug was opened in Bugzilla, we could close the Remedy ticket, because we realistically would never go back to the original complaint and tell the customer who reported it, or when the fix was done.

涵盖该版本的变更请求将自动包含所有软件修复,因此它也遵循变更管理流程。

The change request that covered the release would automatically include all software fixes, so it followed the change management process as well.

——珍妮特

—Janet

如果您的组织正在使用某种流程模型或质量标准管理,请自行学习并与组织中合适的专家合作。保持团队专注于交付可提供真正业务价值的高质量软件,并了解您如何在模型中工作。

If your organization is using some kind of process model or quality standards management, educate yourself about it, and work with the appropriate specialists in your organization. Maintain the team’s focus on delivering high-quality software that provides real business value, and see how you can work within the model.

流程改进模型和框架强调流程的纪律和一致性。很少有软件开发方法比敏捷开发需要更多的纪律。标准只是让您能够衡量您实现目标的进度。敏捷的重点是尽力而为并不断改进。敏捷开发与实现您为自己设定的任何标准或从过程改进测量工具中借用的标准兼容。

Process improvement models and frameworks emphasize discipline and conformance to process. Few software development methodologies require more discipline than agile development. Standards simply enable you to measure your progress toward your goal. Agile’s focus is on doing your best work and constantly improving. Agile development is compatible with achieving whatever standards you set for yourself or borrow from a process improvement measurement tool.

将您的测量目标和标准与改进这些测量的方法分开。设定目标,并了解您需要哪些指标来衡量需要改进的领域是否成功。尝试使用任务卡提供改进的活动,以确保他们获得所需的可见性。

Separate your measurement goals and standards from your means to improve those measurements. Set goals, and know what metrics you need to measure success for areas that need improvement. Try using task cards for activities that provide the improvements in order to ensure they get the visibility they need.

使用现有的质量流程和模型是您在过渡到敏捷开发时可能面临的最大文化问题之一。所有这些改变都是艰难的,但是当你的整个团队都参与进来时,没有什么是不可克服的。

Working with existing quality processes and models is one of the biggest cultural issues you may face as you transition to agile development. All of these changes are hard, but when your whole team gets involved, none are insurmountable.

概括

Summary

在本章中,我们研究了传统的以质量为导向的流程,以及它们如何适应敏捷环境。

In this chapter, we looked at traditional quality-oriented processes and how they can be adapted for an agile environment.

图像正确的指标可以帮助您确保您的团队走上正轨以实现其目标,并为您的投资提供良好的回报。

The right metrics can help you to make sure your team is on track to achieve its goals and provide a good return on your investment in them.

图像指标应该是可见的,提供必要的里程碑来做出决策。

Metrics should be visible, providing necessary milestones upon which to make decisions.

图像使用缺陷跟踪系统的原因包括方便、用作知识库和可追溯性。

The reasons to use a defect tracking system include for convenience, for use as a knowledge base, and for traceability.

图像缺陷跟踪系统经常被用作通信工具,输入和跟踪不必要的错误可能被认为是一种浪费。

Defect tracking systems are too often used as a communication tool, and entering and tracking unnecessary bugs can be considered wasteful.

图像所有的工具,包括DTS,都是需要整个团队使用的,所以在选择工具的时候要多方面考虑。

All tools, including the DTS, need to be used by the whole team, so consider all perspectives when choosing a tool.

图像测试策略是一种长期的整体测试方法,可以放在静态文档中;测试计划应该是项目唯一的。

A test strategy is a long-term overall test approach that can be put in a static document; a test plan should be unique to the project.

图像在盲目接受对特定文件的需求之前考虑替代方案。例如,以小的增量块进行开发的敏捷方法,紧密合作,可能会消除对正式可追溯性文档的需求。将源代码控制系统注释链接到测试可能是另一种方式。

Think about alternatives before blindly accepting the need for specific documents. For example, the agile approach to developing in small, incremental chunks, working closely together, might remove the need for formal traceability documents. Linking the source code control system comments to tests might be another way.

图像传统的质量流程和流程改进模型,例如 SAS 70 审计和 CMMI 标准,可以与敏捷开发和测试共存。团队需要对跳出框框的思维持开放态度,并共同努力解决他们的问题。

Traditional quality processes and process improvement models, such as SAS 70 audits and CMMI standards, can coexist with agile development and testing. Teams need to be open to thinking outside the box and work together to solve their problems.

第三部分敏捷测试象限

Part III The Agile Testing Quadrants

软件质量有很多维度,每个维度都需要不同的测试方法。我们怎么知道我们需要做的所有不同类型的测试?我们如何知道何时“完成”测试?谁进行哪些测试以及如何进行?在这一部分中,我们将解释如何使用敏捷测试象限来确保您的团队涵盖所有需要的测试类别。

Software quality has many dimensions, each requiring a different testing approach. How do we know all the different types of tests we need to do? How do we know when we’re “done” testing? Who does which tests and how? In this part, we explain how to use the Agile Testing Quadrants to make sure your team covers all needed categories of testing.

当然,测试需要工具,我们已经包含了使用工具的示例、有效使用这些工具的策略以及有关何时使用它们的指南。当与为可测试性而设计的代码一起使用时,工具更易于使用。本书的这一部分将讨论这些问题以及更多问题。

Of course, testing requires tools, and we’ve included examples of tools to use, strategies for using those tools effectively, and guidelines about when to use them. Tools are easier to use when used with code that’s designed for testability. These concerns and more are discussed in this part of the book.

第 6 章 测试目的

Chapter 6 The Purpose of Testing

图像

我们为什么要测试?答案似乎很明显,但实际上非常复杂。我们出于很多原因进行测试:发现错误,确保代码可靠,有时只是为了查看代码是否可用。我们进行不同类型的测试以实现不同的目标。软件产品质量有很多组成部分。在本章中,我们将介绍敏捷测试象限。第三部分的其余章节将详细介绍每个象限。敏捷测试象限矩阵帮助测试人员确保他们已经考虑了交付价值所需的所有不同类型的测试。

Why do we test? The answer might seem obvious, but in fact, it’s pretty complex. We test for a lot of reasons: to find bugs, to make sure the code is reliable, and sometimes just to see if the code’s usable. We do different types of testing to accomplish different goals. Software product quality has many components. In this chapter, we introduce the Agile Testing Quadrants. The rest of the chapters in Part III go into detail on each of the quadrants. The Agile Testing Quadrants matrix helps testers ensure that they have considered all of the different types of tests that are needed in order to deliver value.

敏捷测试象限

The Agile Testing Quadrants

第 1 章中,“到底什么是敏捷测试?”,我们针对实现不同目的的不同类别的测试介绍了 Brian Marick 的术语。图 6-1是敏捷测试象限图,显示了四个象限中的每一个如何反映我们测试的不同原因。在一个轴上,我们将矩阵分为支持团队的测试和批评产品的测试。另一个轴将它们分为面向业务和面向技术的测试。

In Chapter 1, “What Is Agile Testing, Anyway?,” we introduced Brian Marick’s terms for different categories of tests that accomplish different purposes. Figure 6-1 is a diagram of the agile testing quadrants that shows how each of the four quadrants reflects the different reasons we test. On one axis, we divide the matrix into tests that support the team and tests that critique the product. The other axis divides them into business-facing and technology-facing tests.

图 6-1敏捷测试象限

Figure 6-1 Agile Testing Quadrants

图像

我们对这些象限进行编号的顺序与不同类型测试的完成时间无关。例如,敏捷开发从客户测试开始,它告诉团队编码什么。各种类型测试的时间取决于每个项目的风险、客户对产品的目标、团队是在使用遗留代码还是在新建项目上工作,以及何时有资源可用于进行测试。

The order in which we’ve numbered these quadrants has no relationship to when the different types of testing are done. For example, agile development starts with customer tests, which tell the team what to code. The timing of the various types of tests depends on the risks of each project, the customers’ goals for the product, whether the team is working with legacy code or on a greenfield project, and when resources are available to do the testing.

支持团队的测试

Tests that Support the Team

左侧的象限包括支持团队开发产品的测试。这种帮助程序员的测试概念对许多测试人员来说是新的,并且是传统项目测试与敏捷项目测试之间的最大区别。在象限 1 和 2 中完成的测试比我们通常认为的测试更多的是需求规范和设计辅助。

The quadrants on the left include tests that support the team as it develops the product. This concept of testing to help the programmers is new to many testers and is the biggest difference between testing on a traditional project and testing on an agile project. The testing done in Quadrants 1 and 2 are more requirements specification and design aids than what we typically think of as testing.

象限 1

左下象限代表测试驱动开发,这是一种核心的敏捷开发实践。

The lower left quadrant represents test-driven development, which is a core agile development practice.

单元测试验证系统的一小部分功能,例如对象或方法。组件测试验证系统较大部分的行为,例如一组提供某些服务的类 [ Meszaros, 2007 ]。这两种类型的测试通常都使用 xUnit 测试自动化工具家族的成员进行自动化。我们将这些测试称为程序员测试、面向开发人员的测试或面向技术的测试。它们使程序员能够衡量 Kent Beck 所说的代码的内部质量 [Beck, 2000, 2004]。

Unit tests verify functionality of a small subset of the system, such as an object or method. Component tests verify the behavior of a larger part of the system, such as a group of classes that provide some service [Meszaros, 2007]. Both types of tests are usually automated with a member of the xUnit family of test automation tools. We refer to these tests as programmer tests, developer-facing tests, or technology-facing tests. They enable the programmers to measure what Kent Beck has called the internal quality of their code [Beck, 2000, 2004].

象限 1 测试的一个主要目的是测试驱动开发 (TDD) 或测试驱动设计。编写测试的过程首先帮助程序员很好地设计他们的代码。这些测试让程序员自信地编写代码来提供故事的功能,而不必担心对系统进行意外更改。他们可以验证他们的设计和架构决策是否合适。单元和组件测试是自动化的,并使用与应用程序相同的编程语言编写。业务专家可能无法通过直接阅读来理解它们,但这些测试并非旨在供客户使用。事实上,内部质量不是与客户协商的;它是由程序员定义的。程序员测试通常是随着每次代码签入而运行的自动化过程的一部分,为团队提供即时、

A major purpose of Quadrant 1 tests is test-driven development (TDD) or test-driven design. The process of writing tests first helps programmers design their code well. These tests let the programmers confidently write code to deliver a story’s features without worrying about making unintended changes to the system. They can verify that their design and architecture decisions are appropriate. Unit and component tests are automated and written in the same programming language as the application. A business expert probably couldn’t understand them by reading them directly, but these tests aren’t intended for customer use. In fact, internal quality isn’t negotiated with the customer; it’s defined by the programmers. Programmer tests are normally part of an automated process that runs with every code check-in, giving the team instant, continual feedback about their internal quality.

象限 2

象限 2 中的测试也支持开发团队的工作,但级别更高。这些面向业务的测试,也称为面向客户的测试和客户测试,定义了外部质量和客户想要的功能。

The tests in Quadrant 2 also support the work of the development team, but at a higher level. These business-facing tests, also called customer-facing tests and customer tests, define external quality and the features that the customers want.

与象限 1 测试一样,它们也推动开发,但在更高的层次上。通过敏捷开发,这些测试源自客户团队提供的示例。他们描述了每个故事的细节。面向业务的测试在功能级别运行,每个测试都验证业务满意度条件。它们是以业务专家可以使用业务领域语言轻松理解的方式编写的。事实上,业务专家使用这些测试来定义产品的外部质量,并且通常帮助编写它们。这个象限可能会重复一些在单元级别完成的测试;然而,象限 2 测试旨在说明和确认更高级别的所需系统行为。

Like the Quadrant 1 tests, they also drive development, but at a higher level. With agile development, these tests are derived from examples provided by the customer team. They describe the details of each story. Business-facing tests run at a functional level, each one verifying a business satisfaction condition. They’re written in a way business experts can easily understand using the business domain language. In fact, the business experts use these tests to define the external quality of the product and usually help to write them. It’s possible this quadrant could duplicate some of the tests that were done at the unit level; however, the Quadrant 2 tests are oriented toward illustrating and confirming desired system behavior at a higher level.

第 8 章,“支持团队的面向业务的测试”,解释了满意度的业务条件。

Chapter 8, “Business-Facing Tests that Support the Team,” explains business conditions of satisfaction.

大多数支持开发团队的面向业务的测试也需要自动化。在这两个象限中进行测试的最重要目的之一是快速提供信息并实现快速故障排除。它们必须经常运行,以便在任何行为意外变化的情况下为团队提供早期反馈。如果可能,这些自动化测试直接在生产代码中的业务逻辑上运行,而无需通过表示层。尽管如此,一些自动化测试必须验证用户界面和客户端应用程序可能使用的任何 API。所有这些测试都应该作为自动化持续集成、构建和测试过程的一部分运行。

Most of the business-facing tests that support the development team also need to be automated. One of the most important purposes of tests in these two quadrants is to provide information quickly and enable fast troubleshooting. They must be run frequently in order to give the team early feedback in case any behavior changes unexpectedly. When possible, these automated tests run directly on the business logic in the production code without having to go through a presentation layer. Still, some automated tests must verify the user interfaces and any APIs that client applications might use. All of these tests should be run as part of an automated continuous integration, build, and test process.

还有另一组测试也属于这个象限。用户交互专家使用模型和线框来帮助与客户验证提议的 GUI(图形用户界面)设计,并在开发人员开始对其进行编码之前将这些设计传达给他们。该组中的测试是帮助支持团队正确构建产品但不是自动化的测试。正如我们将在接下来的章节中看到的那样,象限帮助我们识别我们需要使用的所有不同类型的测试,以帮助推动编码。

There is another group of tests that belongs in this quadrant as well. User interaction experts use mock-ups and wireframes to help validate proposed GUI (graphical user interface) designs with customers and to communicate those designs to the developers before they start to code them. The tests in this group are tests that help support the team to get the product built right but are not automated. As we’ll see in the following chapters, the quadrants help us identify all of the different types of tests we need to use in order to help drive coding.

有些人使用术语“验收测试”来描述象限 2 测试,但我们认为验收测试包含更广泛的测试范围,包括象限 3 和 4。验收测试验证系统的所有方面,包括可用性和性能,满足客户要求。

Some people use the term “acceptance tests” to describe Quadrant 2 tests, but we believe that acceptance tests encompass a broader range of tests that include Quadrants 3 and 4. Acceptance tests verify that all aspects of the system, including qualities such as usability and performance, meet customer requirements.

使用测试来支持团队

象限 1 和 2 自动化测试提供的快速反馈,随着每次代码更改或添加而运行,构成了敏捷团队的基础。这些测试首先指导功能的开发,当自动化时,然后提供安全网以防止重构和新代码的引入导致意外结果。

The quick feedback provided by Quadrants 1 and 2 automated tests, which run with every code change or addition, form the foundation of an agile team. These tests first guide development of functionality, and when automated, then provide a safety net to prevent refactoring and the introduction of new code from causing unexpected results.

丽莎的故事

Lisa’s Story

我们在单独的构建过程中运行支持团队(象限的左半部分)的自动化测试。单元和组件测试在我们的“持续”构建中运行,大约需要八分钟才能完成。尽管程序员在签入之前运行了单元测试,但由于集成问题或环境差异,构建仍然可能失败。一旦我们看到“构建失败”的电子邮件,签入有问题的代码的人就会解决问题。面向业务的功能测试在我们的“完整构建”中运行,它也持续运行,每次签入代码更改时都会启动。它在不到两个小时内完成。这仍然是非常快速的反馈,再一次,构建失败意味着立即采取行动修复问题。有了这些构建作为安全网,我们的代码足够稳定,可以在我们选择的每一天发布迭代。

We run our automated tests that support the team (the left half of the quadrants) in separate build processes. Unit and component tests run in our “ongoing” build, which takes about eight minutes to finish. Although the programmers run the unit tests before they check in, the build might still fail due to integration problems or environmental differences. As soon as we see the “build failed” email, the person who checked in the offending code fixes the problem. Business-facing functional tests run in our “full build,” which also runs continually, kicking off every time a code change is checked in. It finishes in less than two hours. That’s still pretty quick feedback, and again, a build failure means immediate action to fix the problem. With these builds as a safety net, our code is stable enough to release every day of the iteration if we so choose.

——丽莎

—Lisa

象限 1 和 2 中的测试旨在帮助团队交付客户要求的业务价值。他们验证业务逻辑和用户界面的行为是否符合客户提供的示例。软件质量还有其他方面,如果没有技术团队的帮助,客户不会考虑其中的一些方面。产品有竞争力吗?用户界面是否像它需要的那样直观?应用程序安全吗?用户对用户界面的工作方式是否满意?我们需要不同的测试来回答这些类型的问题。

The tests in Quadrants 1 and 2 are written to help the team deliver the business value requested by the customers. They verify that the business logic and the user interfaces behave according to the examples provided by the customers. There are other aspects to software quality, some of which the customers don’t think about without help from the technical team. Is the product competitive? Is the user interface as intuitive as it needs to be? Is the application secure? Are the users happy with how the user interface works? We need different tests to answer these types of questions.

批评产品的测试

Tests that Critique the Product

如果您一直担任客户角色并且必须表达您对软件功能的要求,您就会知道在您看到它之前要确切知道您想要什么是多么困难。即使您对该功能的工作方式有信心,也很难描述它以使程序员完全理解它。

If you’ve been in a customer role and had to express your requirements for a software feature, you know how hard it can be to know exactly what you want until you see it. Even if you’re confident about how the feature should work, it can be hard to describe it so that programmers fully understand it.

“批评”这个词并不具有消极意义。批评可以包括表扬和改进建议。评估软件产品涉及艺术和科学。我们以建设性的方式审查软件,目的是了解我们如何改进它。随着我们的学习,我们可以将新的需求和测试或示例反馈给支持团队和指导开发的流程。

The word “critique” isn’t intended in a negative sense. A critique can include both praise and suggestions for improvement. Appraising a software product involves both art and science. We review the software in a constructive manner, with the goal of learning how we can improve it. As we learn, we can feed new requirements and tests or examples back to the process that supports the team and guide development.

象限 3

面向业务的示例可以帮助团队设计所需的产品,但至少我们的一些示例可能是错误的。业务专家可能会忽略功能,或者如果这不是他们的专业领域,则不会完全正确。团队可能只是误解了一些例子。即使程序员编写了使面向业务的测试通过的代码,他们也可能无法交付客户真正想要的东西。

Business-facing examples help the team design the desired product, but at least some of our examples will probably be wrong. The business experts might overlook functionality, or not get it quite right if it isn’t their field of expertise. The team might simply misunderstand some examples. Even when the programmers write code that makes the business-facing tests pass, they might not be delivering what the customer really wants.

这就是评判第三和第四象限产品的测试发挥作用的地方。象限 3 对运行工作软件的面向业务的测试进行分类,以查看它是否不完全符合预期或无法经受住竞争。当我们进行面向业务的测试来评价产品时,我们会尝试模仿真实用户使用应用程序的方式。这是只有人类才能进行的手动测试。我们可能会使用一些自动化的脚本来帮助我们设置我们需要的数据,但我们必须用我们的感官、我们的大脑和我们的直觉来检查开发团队是否已经交付了客户所需要的商业价值。

That is where the tests to critique the product in the third and fourth quadrants come into play. Quadrant 3 classifies the business-facing tests that exercise the working software to see if it doesn’t quite meet expectations or won’t stand up to the competition. When we do business-facing tests to critique the product, we try to emulate the way a real user would work the application. This is manual testing that only a human can do. We might use some automated scripts to help us set up the data we need, but we have to use our senses, our brains, and our intuition to check whether the development team has delivered the business value required by the customers.

通常,用户和客户执行这些类型的测试。用户验收测试 (UAT) 使客户有机会很好地锻炼新功能并了解他们将来可能需要哪些更改,这是收集新故事创意的好方法。如果您的团队在合同基础上向客户交付软件,则 UAT 可能是批准已完成故事的必要步骤。

Often, the users and customers perform these types of tests. User Acceptance Testing (UAT) gives customers a chance to give new features a good workout and see what changes they may want in the future, and it’s a good way to gather new story ideas. If your team is delivering software on a contract basis to a client, UAT might be a required step in approving the finished stories.

可用性测试是一种测试类型的例子,它有自己的一整套科学。可能会引入焦点小组,在他们使用应用程序时对其进行研究,并进行访谈以收集他们的反应。可用性测试还可以包括从一个页面到另一个页面的导航,甚至是像 Tab 键顺序这样简单的东西。在测试可用性时,了解人们如何使用系统是一个优势。

Usability testing is an example of a type of testing that has a whole science of its own. Focus groups might be brought in, studied as they use the application, and interviewed in order to gather their reactions. Usability testing can also include navigation from page to page or even something as simple as the tabbing order. Knowledge of how people use systems is an advantage when testing usability.

探索性测试是该象限的核心。在探索性测试期间,测试人员同时设计和执行测试,使用批判性思维来分析结果。与脚本测试相比,这提供了更好的了解应用程序的机会。我们不是在谈论即兴和即兴的临时测试。探索性测试是一种比临时测试更周到、更复杂的方法。它以策略为指导,并在定义的约束范围内运行。从每个项目和故事开始,测试人员就开始思考他们想要尝试的场景。随着可测试代码的小块变得可用,测试人员分析测试结果,并且在他们学习的过程中,他们找到新的领域进行探索。探索性测试以与最终用户相同的方式运行系统。测试人员使用他们的创造力和直觉。

Exploratory testing is central to this quadrant. During exploratory testing sessions, the tester simultaneously designs and performs tests, using critical thinking to analyze the results. This offers a much better opportunity to learn about the application than scripted tests. We’re not talking about ad hoc testing, which is impromptu and improvised. Exploratory testing is a more thoughtful and sophisticated approach than ad hoc testing. It is guided by a strategy and operates within defined constraints. From the start of each project and story, testers start thinking of scenarios they want to try. As small chunks of testable code become available, testers analyze test results, and as they learn, they find new areas to explore. Exploratory testing works the system in the same ways that the end users will. Testers use their creativity and intuition. As a result, it is through this type of testing that many of the most serious bugs are usually found.

象限 4

属于第四象限的测试类型对于敏捷开发和任何类型的软件开发一样重要。这些测试是面向技术的,我们以技术而非业务术语来讨论它们。象限 4 中面向技术的测试旨在评判产品特性,例如性能、稳健性和安全性。正如我们将在第 11 章使用面向技术的测试来评价产品”中所述,您的团队已经具备进行这些测试所需的许多技能。例如,程序员可能能够利用多线程引擎将单元测试用于性能测试。但是,创建和运行这些测试可能需要使用专门的工具和额外的专业知识。

The types of tests that fall into the fourth quadrant are just as critical to agile development as to any type of software development. These tests are technology-facing, and we discuss them in technical rather than business terms. Technology-facing tests in Quadrant 4 are intended to critique product characteristics such as performance, robustness, and security. As we’ll describe in Chapter 11, “Critiquing the Product Using Technology-Facing Tests,” your team already possesses many of the skills needed to do these tests. For example, programmers might be able to leverage unit tests into performance tests with a multi-threaded engine. However, creating and running these tests might require the use of specialized tools and additional expertise.

过去,我们听到抱怨说敏捷开发似乎忽略了批评产品的面向技术的测试。这些抱怨可能部分是由于敏捷强调让客户编写故事并确定故事的优先级。非技术客户团队成员通常认为开发人员会处理速度和安全性等问题,而程序员只打算生产客户优先考虑的功能。

In the past, we’ve heard complaints that agile development seems to ignore the technology-facing tests that critique the product. These complaints might be partly due to agile’s emphasis on having customers write and prioritize stories. Nontechnical customer team members often assume that the developers will take care of concerns such as speed and security, and that the programmers are intent on producing only the functionality prioritized by the customers.

如果我们在开始编码之前了解性能、安全性、与其他系统的交互以及其他非功能性属性的要求,那么在设计和编码时考虑到这一点会更容易。其中一些可能比实际功能更重要。例如,如果一个 Internet 零售网站只有一分钟的响应时间,那么客户将不会等待欣赏所有功能都正常工作的事实。在开发周期的每一步都应该考虑对产品进行批评的面向技术的测试,而不是留到最后。在许多情况下,此类测试甚至应该在功能测试之前完成。

If we know the requirements for performance, security, interaction with other systems, and other nonfunctional attributes before we start coding, it’s easier to design and code with that in mind. Some of these might be more important than actual functionality. For example, if an Internet retail website has a one-minute response time, the customers won’t wait to appreciate the fact that all of the features work properly. Technology-facing tests that critique the product should be considered at every step of the development cycle and not left until the very end. In many cases, such testing should even be done before functional testing.

近年来,我们看到许多适用于敏捷开发项目的新轻量级工具可用于支持测试。自动化工具可用于创建测试数据、为手动测试设置测试场景、驱动安全测试并帮助理解结果。对于负载和性能测试等某些工作,自动化是强制性的。

In recent years we’ve seen many new lightweight tools appropriate to an agile development project become available to support tests. Automation tools can be used to create test data, set up test scenarios for manual testing, drive security tests, and help make sense of results. Automation is mandatory for some efforts such as load and performance testing.

当您和您的团队计划新版本或项目时,讨论您需要象限 3 和 4 中的哪些类型的测试,以及应该在何时完成这些测试。不要将负载或可用性测试等基本活动留到最后,否则纠正问题可能为时已晚。

When you and your team plan a new release or project, discuss which types of tests from Quadrants 3 and 4 you need, and when they should be done. Don’t leave essential activities such as load or usability testing to the end, when it might be too late to rectify problems.

使用评价产品的测试

在审查产品的测试过程中产生的信息应该反馈到我们矩阵的左侧,并用于创建新的测试以推动未来的发展。例如,如果服务器在正常负载下出现故障,则需要新的故事和测试来驱动更具可扩展性的架构。使用象限将帮助您计划批评产品的测试以及推动开发的测试。想想你为什么要测试,以确保测试是在开发的最佳阶段进行的。

The information produced during testing to review the product should be fed back into the left side of our matrix and used to create new tests to drive future development. For example, if the server fails under a normal load, new stories and tests to drive a more scalable architecture will be needed. Using the quadrants will help you plan tests that critique the product as well as tests that drive development. Think about why you are testing to make sure that the tests are performed at the optimum stage of development.

敏捷开发的短期迭代让您的团队有机会学习和试验不同的测试象限。如果您发现您的设计无法扩展为时已晚,请尽早开始对下一个故事或项目进行负载测试。如果迭代演示表明团队误解了客户的需求,那么您可能没有做好编写客户测试来指导开发的工作。如果团队推迟了必要的重构,也许单元和组件测试没有提供足够的覆盖率。使用敏捷测试象限来帮助确保在正确的时间完成所有必要的测试。

The short iterations of agile development give your team a chance to learn and experiment with the different testing quadrants. If you find out too late that your design doesn’t scale, start load testing earlier with the next story or project. If the iteration demo reveals that the team misunderstood the customer’s requirements, maybe you’re not doing a good enough job of writing customer tests to guide development. If the team puts off needed refactoring, maybe the unit and component tests aren’t providing enough coverage. Use the agile testing quadrants to help make sure all necessary testing is done at the right time.

知道故事何时完成

Knowing When a Story Is Done

对于大多数产品,我们需要进行所有四类测试才能确信我们提供了正确的价值。并非每个故事都需要进行安全测试,但您不会因为没有想到而忽略它。

For most products, we need all four categories of testing to feel confident we’re delivering the right value. Not every story requires security testing, but you don’t want to omit it because you didn’t think of it.

丽莎的故事

Lisa’s Story

我的团队使用“库存”卡片来确保我们始终考虑所有不同类型的测试。当单元测试还不是一种习惯时,我们为黑板上的每个故事写了一张单元测试卡。我们的“端到端”测试卡提醒程序员完成集成测试工作并确保代码的所有部分协同工作。每个故事也会考虑一张“安全”卡片,如果合适的话,放在板上让每个人都意识到保护数据安全。向客户展示用户界面的任务卡确保我们不会忘记尽早执行此操作,它也有助于我们及早与客户一起开始探索性测试。所有这些卡片帮助我们解决产品质量的所有不同方面。

My team uses “stock” cards to ensure that we always consider all different types of tests. When unit testing wasn’t yet a habit, we wrote a unit test card for each story on the board. Our “end to end” test card reminds the programmers to complete the job of integration testing and to make sure all of the parts of the code work together. A “security” card also gets considered for each story, and if appropriate, put on the board to keep everyone conscious of keeping data safe. A task card to show the user interface to customers makes sure that we don’t forget to do this as early as possible, and it helps us start exploratory testing along with the customers early, too. All of these cards help us address all the different aspects of product quality.

超出单个故事的面向技术的测试在故事板上占有一席之地。我们使用故事来评估负载测试工具并建立性能基线以启动我们的负载和性能测试工作。

Technology-facing tests that extend beyond a single story get their own row on the story board. We use stories to evaluate load test tools and to establish performance baselines to kick off our load and performance-testing efforts.

——丽莎

—Lisa

驱动开发的面向技术和面向业务的测试是敏捷开发的核心,无论您是否真的为它们编写任务卡。它们为您的团队提供了“完成”每个故事的最佳机会。确定执行评估产品的面向技术和面向业务的测试所需的任务可确保您了解产品缺少的内容。来自所有四个象限的测试组合将使团队知道每个特性何时满足客户的功能和质量标准。

The technology-facing and business-facing tests that drive development are central to agile development, whether or not you actually write task cards for them. They give your team the best chance of getting each story “done.” Identifying the tasks needed to perform the technology-facing and business-facing tests that critique the product ensures that you’ll learn what the product is missing. A combination of tests from all four quadrants will let the team know when each feature has met the customer’s criteria for functionality and quality.

共同责任

Shared Responsibility

我们的产品团队需要广泛的专业知识来涵盖所有敏捷测试象限。程序员应该编写支持编程的面向技术的测试,但他们可能在不同时间需要测试人员、数据库设计人员、系统管理员和配置专家的帮助。测试人员与客户一起主要负责面向业务的测试,但程序员参与设计和自动化测试,同时可能会根据需要召集可用性专家和其他专家。第四象限,通过面向技术的测试来评判产品,可能需要更多的专家。无论必须从开发团队外部引入什么资源,该团队仍然负责完成所有四个象限的测试。

Our product teams need a wide range of expertise to cover all of the agile testing quadrants. Programmers should write the technology-facing tests that support programming, but they might need help at different times from testers, database designers, system administrators, and configuration specialists. Testers take primary charge of the business-facing tests in tandem with the customers, but programmers participate in designing and automating tests, while usability and other experts might be called in as needed. The fourth quadrant, with technology-facing tests that critique the product, may require more specialists. No matter what resources have to be brought in from outside the development team, the team is still responsible for getting all four quadrants of testing done.

我们相信,一个成功的团队是每个人都参与产品的制作,并且当出现问题时每个人都分担团队内部的痛苦。实施使我们能够的实践和工具解决所有四个象限的测试有时会很痛苦,但实现成功产品的乐趣是值得付出努力的。

We believe that a successful team is one where everybody participates in the crafting of the product and that everyone shares the team’s internal pain when things go wrong. Implementing the practices and tools that enable us to address all four quadrants of testing can be painful at times, but the joy of implementing a successful product is worth the effort.

管理技术债务

Managing Technical Debt

Ward Cunningham 于 1992 年创造了“技术债务”一词,但我们在软件开发的整个职业生涯中肯定都经历过它!当开发团队走捷径、快速修复或者跳过编写或自动化测试时,技术债务就会增加,因为它处于枪口之下。代码库变得越来越难维护。与金融债务一样,“利息”以更高的维护成本和更低的团队速度的形式复合。程序员害怕做任何改变,更不用说尝试重构来改进代码了,因为害怕破坏它。有时,这种恐惧的存在是因为他们一开始无法理解编码,有时是因为没有测试来发现错误。

Ward Cunningham coined the term “technical debt” in 1992, but we’ve certainly experienced it throughout our careers in software development! Technical debt builds up when the development team takes shortcuts, hacks in quick fixes, or skips writing or automating tests because it’s under the gun. The code base gets harder and harder to maintain. Like financial debt, “interest” compounds in the form of higher maintenance costs and lower team velocity. Programmers are afraid to make any changes, much less attempt refactoring to improve the code, for fear of breaking it. Sometimes this fear exists because they can’t understand the coding to start with, and sometimes it is because there are no tests to catch mistakes.

敏捷测试矩阵中的每个象限都在将技术债务保持在可管理的水平方面发挥作用。支持编码和设计的面向技术的测试有助于保持代码的可维护性。运行单元测试的自动化构建和集成过程是最小化技术债务的必要条件。在编码期间捕获单元级缺陷将使测试人员能够专注于面向业务的测试,以指导团队和改进产品。及时的负载和压力测试让团队知道他们的架构是否能胜任工作。

Each quadrant in the agile testing matrix plays a role in keeping technical debt to a manageable level. Technology-facing tests that support coding and design help keep code maintainable. An automated build and integration process that runs unit tests is a must for minimizing technical debt. Catching unit-level defects during coding will free testers to focus on business-facing tests in order to guide the team and improve the product. Timely load and stress testing lets the teams know whether their architecture is up to the job.

通过花时间并应用资源和实践来将技术债务保持在最低限度,团队将有时间和资源来完成确保产品质量所需的测试。应用敏捷原则在每个级别做好每种类型的测试,反过来又会最大限度地减少技术债务。

By taking the time and applying resources and practices to keep technical debt to a minimum, a team will have time and resources to cover the testing needed to ensure a quality product. Applying agile principles to do a good job of each type of testing at each level will, in turn, minimize technical debt.

在上下文中测试

Testing in Context

我们在敏捷测试矩阵中找到的分类和定义帮助我们确保我们计划并完成我们需要的所有不同类型的测试。但是,我们需要记住,每个组织、产品和团队都有自己独特的情况,每个人都需要根据自己的情况做适合自己的事情。正如 Lisa 的同事 Mike Busse 常说的那样,“这是一种工具,而不是规则。” 随着时间的推移,单个产品或项目的需求可能会发生巨大变化。象限是确保您的团队正在考虑进入“完成”的测试的所有不同方面的有用方法。

Categorizations and definitions such as we find in the agile testing matrix help us make sure we plan for and accomplish all of the different types of testing we need. However, we need to bear in mind that each organization, product, and team has its own unique situation, and each needs to do what works for it in its individual situation. As Lisa’s coworker Mike Busse likes to say, “It’s a tool, not a rule.” A single product or project’s needs might evolve drastically over time. The quadrants are a helpful way to make sure your team is considering all of the different aspects of testing that go into “doneness.”

在为每个故事、迭代和发布计划测试时,我们可以借鉴上下文驱动测试学派的重要原则。

We can borrow important principles from the context-driven school of testing when planning testing for each story, iteration, and release.

图像任何实践的价值都取决于它的背景。

The value of any practice depends on its context.

图像有上下文中的良好实践,但没有最佳实践。

There are good practices in context, but there are no best practices.

图像一起工作的人是任何项目环境中最重要的部分。

People, working together, are the most important part of any project’s context.

图像项目随着时间的推移以通常无法预测的方式展开。

Projects unfold over time in ways that are often not predictable.

图像该产品是一种解决方案。如果问题没有解决,产品就无法工作。

The product is a solution. If the problem isn’t solved, the product doesn’t work.

图像好的软件测试是一个具有挑战性的智力过程。

Good software testing is a challenging intellectual process.

图像只有通过判断力和技巧,在整个项目中协同运用,我们才能在正确的时间做正确的事情,以有效地测试我们的产品。

Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

象限有助于为敏捷测试实践提供背景信息,但您和您的团队必须在进行过程中进行调整。测试人员帮助提供团队调整和更好地工作所需的反馈。使用您的技能在每次迭代和发布中吸引客户。请注意您的团队何时需要超出当前可用范围的角色或知识。

The quadrants help give context to agile testing practices, but you and your team will have to adapt as you go. Testers help provide the feedback the team needs to adjust and work better. Use your skills to engage the customers throughout each iteration and release. Be conscious of when your team needs roles or knowledge beyond what it currently has available.

敏捷测试象限提供了一个清单,以确保您已经覆盖了所有测试基础。检查以下问题的答案:

The Agile Testing Quadrants provide a checklist to make sure you’ve covered all your testing bases. Examine the answers to questions such as these:

图像我们是否使用单元和组件测试来帮助我们为我们的应用程序找到正确的设计?

Are we using unit and component tests to help us find the right design for our application?

图像我们是否有一个自动化的构建过程来运行我们的自动化单元测试以获得快速反馈?

Do we have an automated build process that runs our automated unit tests for quick feedback?

图像我们面向业务的测试是否有助于我们交付符合客户期望的产品?

Do our business-facing tests help us deliver a product that matches customers’ expectations?

图像我们是否捕获了所需系统行为的正确示例?我们需要更多吗?我们的测试是否基于这些示例?

Are we capturing the right examples of desired system behavior? Do we need more? Are we basing our tests on these examples?

图像在开始编码之前,我们是否向用户展示 UI 和报告的原型?用户能否将它们与最终软件的工作方式联系起来?

Do we show prototypes of UIs and reports to the users before we start coding them? Can the users relate them to how the finished software will work?

图像我们是否为探索性测试预留了足够的时间?我们如何处理可用性测试?我们是否让客户参与得足够多?

Do we budget enough time for exploratory testing? How do we tackle usability testing? Are we involving our customers enough?

图像我们是否在开发周期的早期就考虑了性能和安全性等技术要求?我们是否有合适的工具来进行“能力”测试?

Do we consider technological requirements such as performance and security early enough in the development cycle? Do we have the right tools to do “ility” testing?

使用矩阵作为地图开始。进行试验,并使用回顾来不断改进您的工作,以通过测试指导开发,并在您通过测试了解的产品知识的基础上进行构建。

Use the matrix as a map to get started. Experiment, and use retrospectives to keep improving your efforts to guide development with tests and build on what you learn about your product through testing.

概括

Summary

在本章中,我们介绍了敏捷测试象限作为对测试进行分类的便捷方式。四个象限作为指导,以确保产品质量的所有方面都包含在测试和开发过程中。

In this chapter we introduced the Agile Testing Quadrants as a convenient way to categorize tests. The four quadrants serve as guidelines to ensure that all facets of product quality are covered in the testing and developing process.

图像支持团队的测试可用于推动需求。

Tests that support the team can be used to drive requirements.

图像评价产品的测试有助于我们思考应用程序质量的各个方面。

Tests that critique the product help us think about all facets of application quality.

图像使用象限了解您何时完成,并确保整个团队共同负责覆盖矩阵的四个象限。

Use the quadrants to know when you’re done, and ensure the whole team shares responsibility for covering the four quadrants of the matrix.

图像管理技术债务是任何软件开发团队的重要基础。使用象限来考虑不同的维度。

Managing technical debt is an essential foundation for any software development team. Use the quadrants to think about the different dimensions.

图像上下文应始终指导我们的测试工作。

Context should always guide our testing efforts.

第 7 章支持团队的面向技术的测试

Chapter 7 Technology-Facing Tests that Support the Team

图像

我们使用敏捷测试象限作为指南来帮助我们涵盖我们需要的所有测试类型,并帮助我们确保我们拥有正确的资源以在每种类型上取得成功。在本章中,我们将研究第一象限中的测试、支持团队的面向技术的测试,以及支持该测试的工具。该象限中的活动构成了敏捷开发的核心。

We use the Agile Testing Quadrants as a guide to help us cover all the types of testing we need and to help us make sure we have the right resources to succeed at each type. In this chapter, we look at tests in the first quadrant, technology-facing tests that support the team, and at tools to support this testing. The activities in this quadrant form the core of agile development.

敏捷测试基础

An Agile Testing Foundation

我们首先讨论象限 1,因为支持团队的面向技术的测试构成了敏捷开发和测试的基础。请参阅图 7-1以提醒敏捷测试象限,该象限已突出显示。第一象限不仅仅是测试。我们在象限 1 中讨论的单元和组件测试并不是为每个故事编写的第一个测试,但它们有助于指导设计和开发。没有测试驱动的基础设计、自动化单元和组件测试,以及运行测试的持续集成过程,很难及时交付价值。其他象限的所有测试都无法弥补这一象限的不足。我们将在接下来的几章中讨论其他象限,并解释它们是如何组合在一起的。

We discuss Quadrant 1 first because the technology-facing tests that support the team form the foundation of agile development and testing. See Figure 7-1 for a reminder of the Agile Testing Quadrants with this quadrant highlighted. Quadrant 1 is about much more than testing. The unit and component tests we talk about in Quadrant 1 aren’t the first tests written for each story, but they help guide design and development. Without a foundation of test-driven design, automated unit and component tests, and a continuous integration process to run the tests, it’s hard to deliver value in a timely manner. All of the testing in the other quadrants can’t make up for inadequacies in this one. We’ll talk about the other quadrants in the next few chapters and explain how they all fit together.

图 7-1敏捷测试象限,突出显示象限 1

Figure 7-1 The Agile Testing Quadrants, highlighting Quadrant 1

图像

团队需要正确的工具和流程来创建和执行指导开发的面向技术的测试。我们将给出本章最后一节所需工具类型的一些示例。

Teams need the right tools and processes to create and execute technology-facing tests that guide development. We’ll give some examples of the types of tools needed in the last section of this chapter.

象限 1 测试的目的

The Purpose of Quadrant 1 Tests

单元测试和组件测试通过帮助程序员准确理解代码需要做什么以及通过提供指导来确保质量正确的设计。他们帮助团队专注于正在交付的故事,并采取最简单的方法。单元测试验证小到单个对象或方法的部件的行为 [ Meszaros, 2007 ]。组件测试通过测试类或方法之间的交互来帮助巩固系统可部署部分的整体设计。

Unit tests and component tests ensure quality by helping the programmer understand exactly what the code needs to do, and by providing guidance in the right design. They help the team to focus on the story that’s being delivered and to take the simplest approach that will work. Unit tests verify the behavior of parts as small as a single object or method [Meszaros, 2007]. Component tests help solidify the overall design of a deployable part of the system by testing the interaction between classes or methods.

在使用 TDD 时,开发单元测试可能是必不可少的设计工具。当敏捷程序员开始编码任务时,她会编写一个测试来捕获一小段代码的行为,然后处理代码直到测试通过。通过以小的测试-代码-测试增量构建代码,程序员有机会全面考虑客户需要的功能。出现问题时,她可以询问客户。她可以与测试人员配对,以帮助确保该段代码的所有方面及其与其他单元的通信都经过测试。

Developing unit tests can be an essential design tool when using TDD. When an agile programmer starts a coding task, she writes a test that captures the behavior of a tiny bit of code and then works on the code until the test passes. By building the code in small test-code-test increments, the programmer has a chance to think through the functionality that the customer needs. As questions come up, she can ask the customer. She can pair with a tester to help make sure all aspects of that piece of code, and its communication with other units, are tested.

测试驱动开发这个术语会误导那些不明白它更多的是设计而不是测试的从业者。测试先行开发的代码自然是为可测试性而设计的。象限 1 的活动都旨在生产具有尽可能高的内部质量的软件。

The term test-driven development misleads practitioners who don’t understand that it’s more about design than testing. Code developed test-first is naturally designed for testability. Quadrant 1 activities are all aimed at producing software with the highest possible internal quality.

当团队实践 TDD 时,他们会最大限度地减少以后必须发现的错误数量。通过在代码之前编写测试来防止大多数单元级错误。通过编写单元测试来思考设计意味着系统更有可能满足客户的要求。当后期开发测试时间被用于查找和修复程序员测试可能检测到的错误时,就没有时间查找可能对业务产生不利影响的严重问题。从我们的编码过程中泄漏的错误越多,我们的交付就会越慢,最终,质量会受到影响。这就是象限 1 中的程序员测试如此重要的原因。虽然每个团队都应该采用适合其情况的做法,

When teams practice TDD, they minimize the number of bugs that have to be caught later on. Most unit-level bugs are prevented by writing the test before the code. Thinking through the design by writing the unit test means the system is more likely to meet customer requirements. When post-development testing time is occupied with finding and fixing bugs that could have been detected by programmer tests, there’s no time to find the serious issues that might adversely affect the business. The more bugs that leak out of our coding process, the slower our delivery will be, and in the end, it is the quality that will suffer. That’s why the programmer tests in Quadrant 1 are so critical. While every team should adopt practices that work for its situation, a team without these core agile practices is unlikely to benefit much from agile values and principles.

配套基础设施

Supporting Infrastructure

可靠的源代码控制、配置管理和持续集成对于从指导开发的程序员测试中获取价值至关重要。它们使团队始终能够准确了解正在测试的内容。持续集成为我们提供了一种在每次签入新代码时运行测试的方法。当测试失败时,我们知道是谁签入了导致失败的更改,并且那个人可以快速解决问题。连续的集成节省了时间并激励每个程序员在签入新代码之前运行测试。持续集成和构建过程提供了一个可部署的代码包供我们测试。

Solid source code control, configuration management, and continuous integration are essential to getting value from programmer tests that guide development. They enable the team to always know exactly what’s being tested. Continuous integration gives us a way to run tests every time new code is checked in. When a test fails, we know who checked in the change that caused the failure, and that person can quickly fix the problem. Continuous integration saves time and motivates each programmer to run the tests before checking in the new code. A continuous integration and build process delivers a deployable package of code for us to test.

缺乏这些核心敏捷实践的敏捷项目往往会变成“迷你瀑布”。开发周期更短了,但代码仍然被“翻墙”扔给测试人员,因为代码质量很差,他们没有时间进行测试。瀑布这个词不一定是贬义的。我们从事过成功的“瀑布式”项目,在这些项目中,程序员严格自动化单元测试,实践持续集成,并使用自动化构建来运行测试。这些成功的“瀑布式”项目还涉及整个开发周期的客户和测试人员。当我们在没有适当的实践和工具的情况下进行编码时,无论我们如何称呼这个过程,我们都不会及时交付高质量的代码。

Agile projects that lack these core agile practices tend to turn into “mini-waterfalls.” The development cycles are shorter, but code is still being thrown “over the wall” to testers who run out of time to test because the code is of poor quality. The term waterfall isn’t necessarily derogatory. We’ve worked on successful “waterfall” projects where the programmers stringently automate unit tests, practice continuous integration, and use automated builds to run tests. These successful “waterfall” projects also involve customers and testers throughout the development cycle. When we code without appropriate practices and tools, regardless of what we call the process, we’re not going to deliver high-quality code in a timely manner.

为什么要编写和执行这些测试?

Why Write and Execute These Tests?

我们不会在这里详细介绍如何进行 TDD,或者编写单元和组件测试的最佳方法。有几本关于这些主题的优秀书籍。我们的目标是解释为什么这些活动对敏捷测试人员很重要。让我们探讨一些使用支持团队的面向技术的测试的原因。

We’re not going into any details here about how to do TDD, or the best ways to write unit and component tests. There are several excellent books on those subjects. Our goal is to explain why these activities are important to agile testers. Let’s explore some reasons to use technology-facing tests that support the team.

让我们走得更快,做得更多

Lets Us Go Faster and Do More

速度永远不应该是敏捷开发团队的最终目标。试图快速做事并在紧迫的期限内完成任务而不考虑质量会导致我们偷工减料并重拾旧有的坏习惯。如果我们偷工减料,我们就会积累更多的技术债务,并且可能无论如何都会错过最后期限。令人高兴的是,速度是生成具有最高内部质量的代码的长期副作用。持续构建运行单元测试会在问题签入后的几分钟内通知团队失败,并且可以快速找到并修复错误。自动化单元和代码集成测试的安全网使程序员能够经常重构。这使代码保持在合理的可维护性标准,并为投入的时间提供最佳价值。技术债务保持在尽可能低的水平。

Speed should never be the end goal of an agile development team. Trying to do things fast and meet tight deadlines without thinking about the quality causes us to cut corners and revert to old, bad habits. If we cut corners, we’ll build up more technical debt, and probably miss the deadline anyway. Happily, though, speed is a long-term side effect of producing code with the highest possible internal quality. Continuous builds running unit tests notify the team of failure within a few minutes of the problem check-in, and the mistake can be found and fixed quickly. A safety net of automated unit and code integration tests enables the programmers to refactor frequently. This keeps the code at a reasonable standard of maintainability and delivers the best value for the time invested. Technical debt is kept as low as possible.

如果您在一个忽略了单元测试的项目中担任过测试人员,您就会知道将所有时间都花在查找单元级缺陷上是多么容易。在测试“快乐路径”时,您可能会发现如此多的错误,以至于您永远没有时间测试更复杂的场景和边缘情况。随着“查找和修复”周期的拖延,发布截止日期被推迟,或者测试刚刚停止并且有缺陷的产品被强加给毫无戒心的客户。

If you’ve worked as a tester on a project where unit testing was neglected, you know how easy it is to spend all of your time finding unit-level defects. You might find so many bugs while testing the “happy path” that you never have time to test more complex scenarios and edge cases. The release deadline is pushed back as the “find and fix” cycle drags on, or testing is just stopped and a buggy product is foisted off on unsuspecting customers.

与这种情况相比,我们在敏捷团队中的岁月一直是乌托邦。通过测试推动编码实践意味着程序员可能相当了解故事的要求。他们与客户和测试人员进行了广泛的交谈,以阐明所需的行为。各方都了解正在进行的更改。当团队完成编写一个故事的所有任务卡,或一个薄的、可测试的片段时,单元和组件测试已经很好地覆盖了该功能。通常,程序员已经确保故事中至少有一条路径可以从头到尾运行。

Our years on agile teams have been Utopian in contrast to this scenario. Driving coding practices with tests means that the programmers probably understood the story’s requirements reasonably well. They’ve talked extensively with the customers and testers to clarify the desired behaviors. All parties understand the changes being made. By the time the team has completed all of the task cards for coding a story, or a thin, testable slice of one, the feature has been well covered by unit and component tests. Usually the programmers have made sure at least one path through the story works end to end.

这意味着我们作为测试人员几乎不会浪费时间寻找低级错误。我们可能会尝试程序员没有想到的场景,并将时间花在更高级别的业务功能上。设计良好的代码通常是健壮的和可测试的。如果我们发现缺陷,我们会把它展示给程序员,程序员编写单元测试来重现错误,然后快速修复它。实际上,我们有时间专注于探索性测试和其他类型的深入测试,以便对代码进行良好的锻炼,并了解更多关于它应该如何工作的信息。通常,我们发现的唯一“错误”是我们团队中的每个人都错过或误解的要求。如果客户参与并有定期的演示和测试机会,即使是那些也能很快找到。开发团队掌握 TDD 后,

This means that we, as testers, waste little time finding low-level bugs. We’re likely to try scenarios the programmers hadn’t thought of and to spend our time on higher-level business functionality. Well-designed code is usually robust and testable. If we find a defect, we show it to the programmer, who writes a unit test to reproduce the bug and then fixes it quickly. We actually have time to focus on exploratory testing and the other types of in-depth tests to give the code a good workout and learn more about how it should work. Often, the only “bugs” we find are requirements that everyone on our team missed or misunderstood. Even those are found quickly if the customer is involved and has regular demos and test opportunities. After a development team has mastered TDD, the focus for improvement shifts from bug prevention to figuring out better ways to elicit and capture requirements before coding.

关于何时编写测试以及出于什么目的,存在多种不同的理念。尽管敏捷社区普遍认为 TDD 确实可以帮助团队实现质量更好的软件,但每个团队都应就有助于其实现质量目标的方法达成一致。这是程序员测试支持团队的重要方式。让我们看看更多的方法。

There are several different philosophies about when to write tests and for what purpose. It’s up to each team to agree on the approach that helps it achieve its quality objectives, although there is common agreement in the agile community that TDD definitely helps a team achieve better-quality software. This is an important way that programmer tests support the team. Let’s look at some more ways.

使测试人员的工作更轻松

Making Testers’ Jobs Easier

与程序员测试相关的核心实践使许多测试活动更容易完成。程序员在他们自己的沙盒中工作,他们可以在其中测试新代码而不影响其他人的工作。他们不会签入代码,直到代码在他们的沙箱中通过了一套回归测试。

The core practices related to programmer tests make lots of testing activities easier to accomplish. Programmers work in their own sandboxes, where they can test new code without affecting anyone else’s work. They don’t check in code until it has passed a suite of regression tests in their sandbox.

团队考虑测试环境以及测试数据使用什么。单元测试通常使用假对象或模拟对象而不是实际数据库来提高速度,但程序员仍然需要针对真实数据进行测试。测试人员可以帮助他们识别好的测试数据。如果单元测试代表现实生活中的数据,以后发现的问题就会更少。

The team thinks about test environments and what to use for test data. Unit tests usually work with fake or mock objects instead of actual databases for speed, but programmers still need to test against realistic data. Testers can help them identify good test data. If the unit tests represent real-life data, fewer issues will be found later.

丽莎的故事

Lisa’s Story

这是一个小例子。当我现在的团队第一次采用敏捷开发时,我们没有任何自动化测试。我们没有办法生成可部署的代码包,也没有基本的测试环境或测试数据库。我也没有任何办法自己制作一个版本。我们决定开始编写测试先行的代码,并致力于在适当的时候在所有级别进行自动化测试,但我们首先需要一些基础设施。

Here’s a small example. When my current team first adopted agile development, we didn’t have any automated tests. We had no way to produce a deployable code package, and we had no rudimentary test environments or test databases. I didn’t have any means to produce a build myself, either. We decided to start writing code test-first and committed to automating tests at all levels where appropriate, but we needed some infrastructure first.

我们的首要任务是实施连续构建过程,该过程在几天内完成。每个版本都会发送一封电子邮件,其中包含已签入文件的列表和有关更新的评论。我现在可以选择要部署和测试的构建。下一个优先事项是提供独立的测试环境,以便由一个人运行的测试不会干扰其他测试。新的数据库专家创建了新的模式来满足测试需求,并创建了一个规范的、类似生产数据的“种子”数据库。这些模式可以根据需要使用一组干净的数据快速刷新。包括我在内的每个团队成员都有一个独特且独立的测试环境。

Our first priority was to implement a continuous build process, which was done in a couple of days. Each build sent an email with a list of checked-in files and comments about the updates. I could now choose which build to deploy and test. The next priority was to provide independent test environments so that tests run by one person would not interfere with other tests. The new database expert created new schemas to meet testing needs and a “seed” database of canonical, production-like data. These schemas could be refreshed on demand quickly with a clean set of data. Each team member, including me, got a unique and independent test environment.

甚至在团队掌握 TDD 之前,采用的基础架构就已经到位以支持执行测试。这种基础架构使团队能够更有效地开始测试。尝试自动化测试的另一个方面是处理难以测试的遗留应用程序。为启用 TDD 而做出的决定也有助于面向客户的测试。我们决定开始在新架构中重写系统,以促进测试和测试自动化,不仅在单元级别,而且在所有级别。

Even before the team mastered TDD, the adopted infrastructure was in place to support executing tests. This infrastructure enabled the team to start testing much more effectively. Another aspect of trying to automate testing was dealing with a legacy application that was difficult to test. The decisions that were made to enable TDD also helped with customer-facing tests. We decided to start rewriting the system in a new architecture that facilitated testing and test automation, not only at the unit level but at all levels.

——丽莎

—Lisa

编写测试和编写代码时考虑到这些测试意味着程序员总是有意识地使代码可测试。所有这些与基础架构相关的良好品质都会溢出到面向业务的测试和批评产品的测试中。整个团队一直在思考改进设计和简化测试的方法。

Writing tests and writing code with those tests in mind means programmers are always consciously making code testable. All of these good infrastructure-related qualities spill over to business-facing tests and tests that critique the product. The whole team is continually thinking of ways to improve design and make testing easier.

设计时考虑测试

Designing with Testing in Mind

用测试驱动开发的优势之一是编写代码的目的是让测试通过。团队必须从一开始就考虑如何为其编码的每个故事执行和自动化测试。测试驱动开发意味着程序员在编写代码之前会编写每个测试以使其通过。

One advantage of driving development with tests is that code is written with the express intention of making the tests pass. The team has to think, right from the beginning, about how it will execute and automate tests for every story it codes. Test-driven development means that programmers will write each test before they write the code to make it pass.

编写“可测试的代码”是一个简单的概念,但这并不是一件容易的事,尤其是当您正在处理没有自动测试且不是为可测试性而设计的旧代码时。遗留系统通常将业务逻辑、I/O、数据库和用户界面层交织在一起。没有简单的方法来挂钩以在 GUI 下或单元级别自动化测试。

Writing “testable code” is a simple concept, but it’s not an easy task, especially if you’re working on old code that has no automated tests and isn’t designed for testability. Legacy systems often have business logic, I/O, database, and user interface layers intertwined. There’s no easy way to hook in to automate a test below the GUI or at the unit level.

设计可测试架构的常用方法是将应用程序中执行不同功能的不同层分开。理想情况下,您希望使用测试夹具和具有不同输入的测试算法直接访问每一层。为此,您将业务逻辑隔离到它自己的层中,使用假对象而不是尝试访问其他应用程序或实际数据库。如果可以将表示层与底层业务逻辑和数据库访问分离,则可以在不测试底层逻辑的情况下快速测试输入验证。

A common approach in designing a testable architecture is to separate the different layers that perform different functions in the application. Ideally, you would want to access each layer directly with a test fixture and test algorithms with different inputs. To do this, you isolate the business logic into its own layer, using fake objects instead of trying to access other applications or the actual database. If the presentation layer can be separated from underlying business logic and database access, you can quickly test input validation without testing underlying logic.

可测试设计方法的另一个例子是 Alistair Cockburn 的端口和适配器模式 [Cockburn, 2005]。此模式的目的是“创建无需 UI 或数据库即可工作的应用程序,以便您可以针对应用程序运行自动回归测试,在数据库不可用时工作,并将应用程序链接在一起而无需任何用户参与。” 端口接受外部事件,特定技术的适配器将其转换为应用程序可以理解的消息。反过来,应用程序通过端口将输出发送到适配器,适配器创建接收人员或自动用户所需的信号。使用此模式设计的应用程序可以像实际用户一样轻松地由自动化测试脚本驱动。

Another example of an approach to testable design is Alistair Cockburn’s Ports and Adapters pattern [Cockburn, 2005]. This pattern’s intent is to “create your application to work without either a UI or a database so you can run automated regression tests against the application, work when the database becomes unavailable, and link applications together without any user involvement.” Ports accept outside events, and a technology-specific adapter converts it into a message that can be understood by the application. In turn, the application sends output via a port to an adapter, which creates the signals needed by the receiving human or automated users. Applications designed using this pattern can be driven by automated test scripts as easily as by actual users.

有关 Alastair Cockburn 的端口和适配器模式的更多信息,请参阅参考书目。

See the bibliography for more information on Alastair Cockburn’s Ports and Adapters pattern.

更明显的是如何在未开发项目上编写测试先行的代码。自动化单元测试未涵盖的遗留系统提出了巨大的挑战。很难为不是为可测试性而设计的代码编写单元测试,也很难更改不受单元测试保护的代码。很多球队都有遵循 Michael Feathers 在Working Effectively with Legacy Code [Feathers, 2004]中解释的“遗留代码救援”技术。其他团队,例如 Lisa 的团队,旨在“扼杀”他们的遗留代码。该策略源于 Martin Fowler 的“扼杀器应用程序”[ Fowler, 2004 ]。在仍然维护旧系统的同时,新故事在新架构中以测试优先的方式编码。随着时间的推移,系统的大部分已转换为新架构,目标是最终废除旧系统。

It’s more obvious how to code test-first on a greenfield project. Legacy systems, which aren’t covered by automated unit tests, present a huge challenge. It’s hard to write unit tests for code that isn’t designed for testability, and it’s hard to change code that isn’t safeguarded with unit tests. Many teams have followed the “legacy code rescue” techniques explained by Michael Feathers in Working Effectively with Legacy Code [Feathers, 2004]. Other teams, such as Lisa’s, aim to “strangle” their legacy code. This strategy stems from Martin Fowler’s “strangler application” [Fowler, 2004]. New stories were coded test-first in a new architecture while the old system was still maintained. Over time, much of the system has been converted to the new architecture, with the goal of eventually doing away with the old system.

参考书目有更多关于“拯救”和“扼杀”遗留代码方法的文章的链接。

The bibliography has links to more articles about “rescue” and “strangler” approaches to legacy code.

在遗留大型机类型的环境中进行敏捷测试会带来特殊的挑战,其中最重要的是缺乏有关如何成功进行测试的出版物和信息。COBOL、大型机及其同类产品仍在广泛使用。当您寻找在应用程序中启用自动化测试的方法时,让敏捷原则和价值观指导您的团队。您可能需要调整一些技巧;例如,也许你不能写代码测试先行,但你可以在写完代码后马上进行测试。当需要解决的是团队的问题,而不仅仅是测试人员的问题时,您就会找到编写测试的方法。

Agile testing in a legacy mainframe type of environment presents particular challenges, not the least of which is the lack of availability of publications and information about how to do it successfully. COBOL, mainframes, and their ilk are still widely used. Let agile principles and values guide your team as you look for ways to enable automated testing in your application. You might have to adapt some techniques; for example, maybe you can’t write code test-first, but you can test soon after writing the code. When it’s the team’s problem to solve, and not just the testers’ problem, you’ll find a way to write tests.

有关 RPGUnit 的更多信息,请参阅www.RPGUnit.org

For more about RPGUnit, see www.RPGUnit.org.

您的团队可以找到适合您的可测试性设计方法。秘诀在于整个团队对测试和质量的承诺。当一个团队不断努力编写测试并使它们通过时,它就会找到完成它的方法。团队应该花时间考虑他们如何创建一个架构,使自动化测试易于创建、维护成本低且寿命长。如果自动化测试没有为投资回报足够的价值,不要害怕重新审视架构。

Your team can find an approach to designing for testability that works for you. The secret is the whole-team commitment to testing and quality. When a team is constantly working to write tests and make them pass, it finds a way to get it done. Teams should take time to consider how they can create an architecture that will make automated tests easy to create, inexpensive to maintain, and long-lived. Don’t be afraid to revisit the architecture if automated tests don’t return enough value for the investment in them.

及时反馈

Timely Feedback

单元测试的最大价值在于其反馈的速度。我们认为,运行单元测试的持续集成和构建过程应该在十分钟内完成。如果每个程序员每天检查代码几次,那么较长的构建和测试过程将导致更改开始堆积。作为测试人员,不得不长时间等待新功能或错误修复可能会令人沮丧。如果出现编译错误或单元测试失败,延迟会变得更糟,尤其是快到回家的时候了!

The biggest value of unit tests is in the speed of their feedback. In our opinion, a continuous integration and build process that runs the unit tests should finish within ten minutes. If each programmer checks code in several times a day, a longer build and test process will cause changes to start stacking up. As a tester, it can be frustrating to have to wait a long time for new functionality or a bug fix. If there’s a compile error or unit test failure, the delay gets even worse, especially if it’s almost time to go home!

在单元级别之上运行测试的构建和测试过程(例如功能性 API 测试或 GUI 测试)将花费更长的时间。至少有一个快速运行的构建过程,第二个运行较慢的测试。应该至少有一个每日“构建”来运行所有较慢的功能测试。然而,即使那样也可能很笨拙。当测试失败并且问题得到解决时,需要多长时间才能确定构建再次通过?

A build and test process that runs tests above the unit level, such as functional API tests or GUI tests, is going to take longer. Have at least one build process that runs quickly, and a second that runs the slower tests. There should be at least one daily “build” that runs all of the slower functional tests. However, even that can be unwieldy. When a test fails and the problem is fixed, how long will it take to know for sure that the build passes again?

如果您的构建和测试过程花费的时间太长,请让您的团队分析减速的原因并采取措施加快构建速度。这里有一些例子。

If your build and test process takes too long, ask your team to analyze the cause of the slowdown and take steps to speed up the build. Here are a few examples.

图像数据库访问通常会消耗大量时间,因此请考虑在可能的情况下使用假对象来替换数据库,尤其是在单元级别。

Database access usually consumes lots of time, so consider using fake objects, where possible, to replace the database, especially at the unit level.

图像将运行时间较长的集成和数据库访问测试移至辅助构建和测试过程。

Move longer-running integration and database-access tests to the secondary build and test process.

图像查看测试是否可以并行运行,以便它们更快地完成。

See if tests can run in parallel so that they finish faster.

图像 运行回归测试系统所需的最少测试。

Run the minimum tests needed for regression testing your system.

图像在多台构建机器上分配任务。

Distribute tasks across multiple build machines.

图像升级运行构建的硬件和软件。

Upgrade the hardware and software that run the build.

图像找到花费最多时间的区域并采取渐进的步骤来加快速度。

Find the area that takes the most time and take incremental steps to speed it up.

丽莎的故事

Lisa’s Story

在我当前团队敏捷演化的早期,我们几乎没有进行单元测试,因此我们在持续构建中包含了一些 GUI 冒烟测试,这些测试在每次签入源代码控制系统时启动。当我们有足够多的单元测试可以很好地了解代码何时被破坏时,我们将 GUI 测试和 FitNesse 功能测试移动到一个单独的构建和测试过程中,该过程在晚上运行,与我们的持续构建在同一台机器上。

Early in my current team’s agile evolution, we had few unit tests, so we included a few GUI smoke tests in our continual build, which kicked off on every check-in to the source code control system. When we had enough unit tests to feel good about knowing when code was broken, we moved the GUI tests and the FitNesse functional tests into a separate build and test process that ran at night, on the same machine as our continual build.

我们持续不断的构建开始只用了不到 10 分钟,但很快就需要超过 15 分钟才能完成。我们编写了任务卡来诊断和解决问题。程序员早期编写的单元测试设计得不好,因为没有人确定编写单元测试的最佳方式。预算时间用于重构单元测试,使用模拟数据访问对象而不是真实数据库,并重新设计测试以提高速度。这使构建时间达到了大约八分钟。每次它开始蔓延时,我们都会通过重构、删除不必要的测试、升级硬件以及选择不同的软件来帮助构建运行得更快来解决问题。

Our continual ongoing build started out taking less than 10 minutes, but soon was taking more than 15 minutes to complete. We wrote task cards to diagnose and fix the problem. The unit tests that the programmers had written early on weren’t well designed, because nobody was sure of the best way to write unit tests. Time was budgeted to refactor the unit tests, use mock data access objects instead of the real database, and redesign tests for speed. This got the build to around eight minutes. Every time it has started to creep up, we’ve addressed the problem with refactoring, removing unnecessary tests, upgrading the hardware, and choosing different software that helped the build run faster.

随着我们的功能测试涵盖更多代码,夜间构建更频繁地崩溃。因为夜间构建与持续进行的构建在同一台机器上运行,所以验证构建再次“绿色”的唯一方法是停止正在进行的构建,这消除了我们的快速反馈。这开始浪费大家的时间。我们为更长的构建购买并设置了另一台构建机器,现在它也可以连续运行。这比花费这么多时间在同一台机器上运行两个构建要便宜得多,现在我们也能从我们的功能测试中得到快速反馈。

As our functional tests covered more code, the nightly build broke more often. Because the nightly build ran on the same machine as the continual ongoing one, the only way to verify that the build was “green” again was to stop the ongoing build, which removed our fast feedback. This started to waste everyone’s time. We bought and set up another build machine for the longer build, which now also runs continuously. This was much less expensive than spending so much time keeping two builds running on the same machine, and now we get quick feedback from our functional tests as well.

——丽莎

—Lisa

哇,多个连续的构建和测试过程提供持续的反馈——这对很多测试人员来说听起来像是一个梦想。回归错误将在修复成本最低的时候及早发现。这是编写面向技术的测试的重要原因。但是,我们会不会对他们太着迷了?让我们看看面向技术的测试和面向业务的测试之间的界限。

Wow, multiple continuous build and test processes providing constant feedback—it sounds like a dream to a lot of testers. Regression bugs will be caught early, when they’re cheapest to fix. This is a great reason for writing technology-facing tests. Can we get too carried away with them, though? Let’s look at the line between technology-facing tests and business-facing tests.

面向技术的测试在哪里停止?

Where Do Technology-Facing Tests Stop?

我们经常听到人们担心面向客户的测试会与面向技术的测试重叠太多,以至于团队会浪费时间。我们知道面向业务的测试可能涵盖与单元或代码集成测试相同的领域,但它们的目的如此不同,因此不必担心浪费。

We often hear people worry that the customer-facing tests will overlap so much with the technology-facing tests that the team will waste time. We know that business-facing tests might cover a bit of the same ground as unit or code integration tests, but they have such different purposes that waste isn’t a worry.

例如,我们有一个故事来计算贷款分期偿还时间表并将其显示给正在申请贷款的用户。这个故事的单元测试可能会测试非法参数,例如,如果业务不允许,则每年支付频率。可能有一个单元测试来确定预期的贷款支付开始日期,给定金额、利率、开始日期和频率的一些定义。单元级测试可以涵盖支付频率、金额、利息日期、期限和开始日期的不同组合,以证明摊销计算是正确的。它们可以涵盖闰年等情景。当这些测试通过时,程序员会对代码充满信心。

For example, we have a story to calculate a loan amortization schedule and display it to a user who’s in the process of requesting a loan. A unit test for this story would likely test for illegal arguments, such as an annual payment frequency if the business doesn’t allow it. There might be a unit test to figure the anticipated loan payment start date given some definition of amount, interest rate, start date, and frequency. Unit-level tests could cover different combinations of payment frequency, amount, interest date, term, and start date in order to prove that the amortization calculation is correct. They could cover scenarios such as leap years. When these tests pass, the programmer feels confident about the code.

每个单元测试都是独立的,一次测试一个维度。这意味着当单元测试失败时,程序员可以快速识别问题并尽快解决问题。面向业务的测试很少只涵盖一个维度,因为它们是从业务角度处理的。

Each unit test is independent and tests one dimension at a time. This means that when a unit test fails, the programmer can identify the problem quickly and solve the issue just as quickly. The business-facing tests very seldom cover only one dimension, because they are tackled from a business point of view.

这个故事的面向业务的测试将为业务规则、用户界面中的表示和错误处理定义更多细节。他们将验证支付细节,例如应用的本金和利息,是否正确显示在用户界面中。他们将测试用户界面上每个字段的验证,并指定针对余额不足或不合格等情况的错误处理。他们可以测试管理员在同一天处理两笔贷款支付的场景,这可能更难在单位级别模拟。

The business-facing tests for this story would define more details for the business rules, the presentation in the user interface, and error handling. They would verify that payment details, such as the principal and interest applied, display correctly in the user interface. They would test validations for each field on the user interface, and specify error handling for situations such as insufficient balance or ineligibility. They could test a scenario where an administrator processes two loan payments on the same day, which might be harder to simulate at the unit level.

面向业务的测试涵盖更复杂的用户场景,验证最终用户是否有良好的体验。尽可能将测试推到较低的级别;如果您确定了一个可以在单元级别自动化的测试用例,那几乎总能获得更好的投资回报。

The business-facing tests cover more complex user scenarios and verify that the end user will have a good experience. Push tests to lower levels whenever possible; if you identify a test case that can be automated at the unit level, that’s almost always a better return on investment.

第 13 章,“为什么我们想要自动化测试以及是什么阻碍了我们”,更多地讨论了不同类型测试的投资回报率。

Chapter 13, “Why We Want to Automate Tests and What Holds Us Back,” talks more about the ROI of the different types of tests.

如果涉及应用程序的多个区域或层,则可能无法在单元级别实现自动化。面向技术和面向业务的级别都可能在第一次贷款支付日期前后进行测试,但它们出于不同的原因进行测试。单元测试将检查日期的计算,面向业务的测试将验证它是否正确显示在借款人的贷款报告中。

If multiple areas or layers of the application are involved, it might not be possible to automate at the unit level. Both technology-facing and business-facing levels might have tests around the date of the first loan payment, but they check for different reasons. The unit test would check the calculation of the date, and the business-facing test would verify that it displays correctly in the borrower’s loan report.

学习编写象限 1 测试很难。许多过渡到敏捷开发的团队开始时没有自动化单元测试,甚至没有持续集成和构建过程。在下一节中,我们建议敏捷测试人员在他们的团队不处理象限 1 测试时可以采取的行动。

Learning to write Quadrant 1 tests is hard. Many teams making the transition to agile development start out with no automated unit tests, not even a continuous integration and build process. In the next section, we suggest actions agile testers can take if their teams don’t tackle Quadrant 1 tests.

如果团队不做这些测试怎么办?

What If the Team Doesn’t Do These Tests?

许多组织已经决定尝试敏捷开发,或者至少表明了这种意图,但并不了解如何进行成功的转换。当我们担任测试人员角色时,我们可以做些什么来帮助开发团队实施 TDD、持续集成和其他成功开发的关键实践?

Many an organization has decided to try agile development, or at least stated that intention, without understanding how to make a successful transition. When we’re in a tester role, what can we do to help the development team implement TDD, continuous integration, and other practices that are key to successful development?

多年来我们的经验是,如果我们自己不是程序员,那么在敦促程序员采用 TDD 等实践时,我们不一定有多少可信度。如果我们可以坐下来向他们展示如何编写测试先行的代码,那将是有说服力的,但我们中的许多测试人员都没有那种经验。我们还发现传福音不起作用。从概念上来说服某人 TDD 是个好主意并不难。帮助他们获得牵引力实际编码测试先行要困难得多。

Our experience over the years has been that if we aren’t programmers ourselves, we don’t necessarily have much credibility when we urge the programmers to adopt practices such as TDD. If we could sit down and show them how to code test-first, that would be persuasive, but many of us testers don’t have that kind of experience. We’ve also found that evangelizing doesn’t work. It’s not that hard to convince someone conceptually that TDD is a good idea. It’s much trickier to help them get traction actually coding test-first.

测试人员可以做什么?

What Can Testers Do?

如果您是所谓的“敏捷”团队中的测试人员,该团队甚至不执行单元测试自动化或生成连续构建(或者至少不每天进行构建),您很快就会感到沮丧。不要放弃;继续集思广益,寻找一种方法来推动积极的转变。尝试利用社交时间或其他放松活动来花一些时间,看看您可以产生哪些新想法来让所有团队成员参与进来。

If you’re a tester on a so-called “agile” team that isn’t even automating unit tests or producing continuous builds—or at a minimum, doing builds on a daily basis—you’re going to get frustrated pretty quickly. Don’t give up; keep brainstorming for a way to get traction on a positive transition. Try using social time or other relaxing activity to take some quality time to see what new ideas you can generate to get all team members on board.

要避免的一个陷阱是让测试人员编写单元测试。因为 TDD 实际上更像是一种设计活动,所以编写代码的人在编写代码之前也必须编写测试。程序员还需要自动化单元测试提供的即时反馈。在编写代码后由其他人编写的单元测试可能仍然可以防止回归缺陷,但它们不会获得程序员编写的测试的最有价值的好处。

One trap to avoid is having testers write the unit tests. Because TDD is reallymore of a design activity, it’s essential that the person writing the code also write the tests, before writing the code. Programmers also need the immediate feedback that automated unit tests give. Unit tests written by someone else after the code is written might still guard against regression defects, but they won’t have the most valuable benefits of tests written by the programmer.

丽莎的故事

Lisa’s Story

每当我想要影响变化时,我都会求助于Mary Lynn Manns 和 Linda Rising [ 2004 年] 所著的Fearless Change中的模式。在两个 XP 团队工作之后,我加入了一个声称想要变得敏捷但没有朝着可靠的开发实践迈进的团队。我在Fearless Change中发现了几种模式,试图将团队推向敏捷实践。

Whenever I’ve wanted to effect change, I’ve turned to the patterns in Fearless Change by Mary Lynn Manns and Linda Rising [2004]. After working on two XP teams, I joined a team that professed a desire to become agile but wasn’t making strides toward solid development practices. I found several patterns in Fearless Change to try to move the team toward agile practices.

“寻求帮助”是一种对我有帮助的模式。这种模式部分表示:“由于将新想法引入组织的任务是一项艰巨的任务,因此寻找人员和资源来帮助您的努力” [ Manns and Rising, 2004 ]。当我希望我的团队开始使用 FitNesse 时,我确定了最支持我的事业的程序员,并请他与我结对,为他正在处理的故事编写 FitNesse 测试。他告诉其他程序员他从 FitNesse 测试中获得的好处,这鼓励他们也尝试一下。大多数人都想提供帮助,而敏捷就是团队一起工作,所以没有理由单打独斗。

“Ask for Help” was one pattern that helped me. This pattern says, in part: “Since the task of introducing a new idea into an organization is a big job, look for people and resources to help your efforts” [Manns and Rising, 2004]. When I wanted my team to start using FitNesse, I identified the programmer who was most sympathetic to my cause and asked him to pair with me to write FitNesse tests for the story he was working on. He told the other programmers about the benefits he derived from the FitNesse tests, which encouraged them to try it too. Most people want to help, and agile is all about the team working together, so there’s no reason to go it alone.

“棕色包”是我的团队充分利用的另一种变化模式。例如,我现在的团队举行了几次 brown bag 会议,他们一起编写单元测试。“Guru on Your Side”是一种富有成效的模式,在这种模式中,您可以寻求一位受人尊敬的团队成员的帮助,他可能理解您想要实现的目标。我以前所在的团队没有动力编写单元测试。团队中最有经验的程序员同意我的观点,即测试驱动开发是一个好主意,他为团队的其他成员树立了榜样。

“Brown Bag” is another change pattern that my teams have put to good use. For example, my current team held several brown bag sessions where they wrote unit tests together. “Guru on Your Side” is a productive pattern in which you enlist the help of a well-respected team member who might understand what you’re trying to achieve. A previous team I was on was not motivated to write unit tests. The most experienced programmer on the team agreed with me that test-driven development was a good idea, and he set an example for the rest of the team.

我们认为您会发现敏捷团队中总会有人同情您的事业。争取那个人的支持,尤其是当团队认为他或她是高级大师时。

We think you’ll find that there’s always someone on an agile team who’s sympathetic to your cause. Enlist that person’s support, especially if the team perceives him or her as a senior-level guru.

——丽莎

—Lisa

作为敏捷团队的测试人员,您可以做很多事情来充当变更推动者,但您的潜在影响是有限的。在某些情况下,强大的管理支持是推动团队从事第一象限活动的关键。

As a tester on an agile team, there’s a lot you can do to act as a change agent, but your potential impact is limited. In some cases, strong management support is the key to driving the team to engage in Quadrant 1 activities.

管理者能做什么?

What Can Managers Do?

如果您正在管理一个开发团队,您可以做很多事情来鼓励测试驱动开发和单元测试自动化。与产品负责人合作,将质量作为您的目标,并将质量标准传达给团队。鼓励程序员花时间把工作做到最好,而不是担心赶不上最后期限。如果交货日期处于危险之中,则推动缩小范围,而不是质量。你的工作是向业务经理解释将质量放在首位将如何确保他们获得最佳业务价值。

If you’re managing a development team, you can do a lot to encourage test-driven development and unit test automation. Work with the product owner to make quality your goal, and communicate the quality criteria to the team. Encourage the programmers to take time to do their best work instead of worrying about meeting a deadline. If a delivery date is in jeopardy, push to reduce the scope, not the quality. Your job is to explain to the business managers how making quality a priority will ensure that they get optimum business value.

给团队时间学习,并提供专业的实践培训。引入经验丰富的敏捷开发教练或聘请具有使用这些实践经验的人,他们可以将这些技能传授给团队的其他成员。为重大重构、就编写单元和代码集成测试的最佳方法集思广益以及评估、安装和升级工具安排时间。测试经理应该与开发一起工作管理人员鼓励增强可测试性并允许测试人员编写可执行测试的实践。测试经理还可以确保测试人员有时间学习如何使用团队决定实施的自动化工具和框架。

Give the team time to learn, and provide expert, hands-on training. Bring in an experienced agile development coach or hire someone with experience in using these practices who can transfer those skills to the rest of the team. Budget time for major refactoring, for brainstorming about the best approach to writing unit and code integration tests, and for evaluating, installing, and upgrading tools. Test managers should work with development managers to encourage practices that enhance testability and allow testers to write executable tests. Test managers can also make sure testers have time to learn how to use the automation tools and frameworks that the team decides to implement.

这是一个团队问题

It’s a Team Problem

虽然您可以找到成为有效变革推动者的方法,但最好的办法是让整个团队参与解决问题。如果您还没有在每次迭代后进行回顾,建议尝试这种做法或其他类型的流程改进。在回顾中,提出阻碍成功交付的问题。例如,“我们没有在迭代结束前完成测试任务”是整个团队要解决的问题。如果未完成的原因之一是大量的单元级错误,建议尝试使用 TDD,但允许程序员提出他们自己的解决问题的方法。鼓励团队尝试一种新方法进行几次迭代,看看它是如何工作的。

While you can find ways to be an effective change agent, the best thing to do is involve the whole team in solving the problems. If you aren’t already doing retrospectives after every iteration, propose trying this practice or some other type of process improvement. At the retrospective, raise issues that are hampering successful delivery. For example, “We aren’t finishing testing tasks before the end of the iteration” is a problem for the whole team to address. If one reason for not finishing is the high number of unit-level bugs, suggest experimenting with TDD, but allow programmers to propose their own ways to address the problem. Encourage the team to try a new approach for a few iterations and see how it works.

有关回顾和过程改进的更多信息,请参阅第 19 章结束迭代”。

More about retrospectives and process improvement in Chapter 19, “Wrap Up the Iteration.”

支持团队开发过程的面向技术的测试是所有需要进行的测试的重要基础。如果团队在这个象限中的测试做得不够好,那么其他类型的测试就会困难得多。这并不意味着你不能从其他象限中单独获得价值——这只是意味着这样做会更难,因为团队的代码将缺乏内部质量,一切都将花费更长的时间。

Technology-facing tests that support the team’s development process are an important foundation for all of the testing that needs to happen. If the team isn’t doing an adequate job with the tests in this quadrant, the other types of testing will be much more difficult. This doesn’t mean you can’t get value from the other quadrants on their own—it just means it will be harder to do so because the team’s code will lack internal quality and everything will take longer.

没有正确的工具和基础设施,就无法完成面向技术的测试。在下一节中,我们将查看团队需要有效进行象限 1 测试的工具类型的示例。

Technology-facing tests can’t be done without the right tools and infrastructure. In the next section, we look at examples of the types of tools a team needs to be effective with Quadrant 1 tests.

工具包

Toolkit

没有什么神奇的工具可以确保成功。然而,工具可以帮助优秀的人把工作做到最好。建立正确的基础架构以支持面向技术的测试至关重要。有大量优秀的工具可供选择,而且它们一直在改进。您的团队必须找到最适合您情况的工具。

There’s no magical tool that will ensure success. However, tools can help good people do their best work. Building up the right infrastructure to support technology-facing tests is critical. There’s a huge selection of excellent tools available, and they improve all the time. Your team must find the tools that work best for your situation.

源代码控制

Source Code Control

源代码控制也有其他名称,例如版本控制或修订控制。它当然不是新的,也不是敏捷开发所独有的,但是没有它,任何软件开发团队都无法成功。这就是我们在这里讨论它的原因。如果没有源代码控制,您将永远无法确定您正在测试的是什么。程序员是只更改了他说他更改的模块,还是忘记了他对其他模块所做的更改?如果没有某种版本控制系统,您将无法撤销不需要的或错误的更改。源代码控制可以防止不同的程序员在同一模块上互相修改。如果没有版本控制,您将无法确定将哪些代码发布到生产环境中。

Source code control is known by other names too, such as version control or revision control. It’s certainly not new, or unique to agile development, but no software development team can succeed without it. That’s why we’re discussing it here. Without source code control, you’ll never be sure what you’re testing. Did the programmer change only the module he said he changed, or did he forget changes he made to other modules? You can’t back out unwanted or erroneous changes without some kind of versioning system. Source code control keeps different programmers from walking on each other’s changes to the same modules. Without versioning, you can’t be sure what code to release to production.

Software Configuration Management Patterns: Effective Teamwork, Practical Integrations [ 2003 ],作者 Stephen Berczuk 和 Brad Appleton,是学习如何以及为什么使用源代码控制的好资源。源代码控制对于任何风格的软件开发都是必不可少的。

Software Configuration Management Patterns: Effective Teamwork, Practical Integrations [2003], by Stephen Berczuk and Brad Appleton, is a good resource to use to learn how and why to use source code control. Source code control is essential to any style of software development.

也对自动化测试脚本使用源代码控制。将自动化测试与他们测试的相应代码版本联系起来很重要,以防您将来需要针对该版本重新运行测试。当您标记或标记构建时,请确保您也标记或标记测试代码,即使它没有发布到生产环境。

Use source code control for automated test scripts, too. It’s important to tie the automated tests with the corresponding code version that they tested in case you need to rerun tests against that version in the future. When you label or tag a build, make sure you label or tag the test code too, even if it doesn’t get released to production.

团队可以组织他们的代码层次结构,为生产代码、相应的单元测试和更高级别的测试脚本提供存储库。这样做可能需要一些头脑风暴和试验才能获得正确的结构。

Teams can organize their code hierarchy to provide a repository for production code, corresponding unit tests, and higher-level test scripts. Doing this might require some brainstorming and experimenting in order to get the right structure.

有许多很棒的选项可供选择。CVS 和 Subversion (SVN) 等开源系统易于实施,与持续构建过程和 IDE 集成,并且非常健壮。IBM Rational ClearCase 和 Perforce 等供应商工具可能会添加一些功能来补偿它们经常带来的增加的开销。

There are many terrific options to choose from. Open source systems such as CVS and Subversion (SVN) are easy to implement, integrate with a continuous build process and IDEs, and are robust. Vendor tools such as IBM Rational ClearCase and Perforce might add features that compensate for the increased overhead they often bring.

源代码控制与开发环境紧密集成。让我们看一下敏捷团队使用的一些 IDE。

Source code control is tightly integrated with development environments. Let’s look at some IDEs used by agile teams.

集成开发环境

IDEs

一个好的IDE(集成开发环境)可以帮助敏捷团队中的程序员和测试人员。IDE 与源代码控制系统集成,有助于防止版本控制问题和更改相互影响。IDE 内的编辑器特定于编程语言,即使在您编写代码时也会标记错误。最重要的是,IDE 提供了对重构的支持。

A good IDE (integrated development environment) can be helpful for programmers and testers on an agile team. The IDE integrates with the source code control system to help prevent problems with versioning and changes walking on each other. The editors inside an IDE are specific to the programming language and flag errors even as you write the code. Most importantly, IDEs provide support for refactoring.

使用 IDE 的程序员往往具有强烈的个人偏好。但是,有时组织会规定所有程序员都必须使用特定的 IDE。这可能是因为许可,也可能是为了鼓励开放结对编程。如果另一个程序员使用相同的 IDE,则与另一个程序员配对会更容易,但通常不必使用相同的 IDE。大多数工具的工作原理相似,因此不难从一种 IDE 更改为另一种 IDE 以满足新需求或利用新功能。一些顽固分子仍然更喜欢使用经过验证的技术,例如 vi、vim 或带有 make 文件的 emacs 而不是 IDE。

Programmers who use an IDE tend to have strong personal preferences. However, sometimes an organization decrees that all programmers must use a specific IDE. This might be because of licensing, or it might be intended to encourage open pair programming. It is easier to pair with another programmer if the other person uses the same IDE, but it’s generally not essential for the same one to be used. Most tools work similarly, so it’s not hard to change from one IDE to another in order to meet new needs or take advantage of new features. Some diehards still prefer to use tried-and-true technology such as vi, vim, or emacs with make files rather than an IDE.

Eclipse 和 NetBeans 等开源 IDE 以及 Visual Studio 和 IntelliJ IDEA 等专有系统被敏捷团队广泛使用。IDE 具有支持不同语言和工具的插件。它们在测试脚本中的表现与在生产代码中的表现一样好。

Open source IDEs such as Eclipse and NetBeans are widely used by agile teams, along with proprietary systems such as Visual Studio and IntelliJ IDEA. IDEs have plug-ins to support different languages and tools. They work as well with test scripts as they do with production code.

丽莎的故事

Lisa’s Story

在我目前的团队中,一些程序员使用 IntelliJ IDEA,而其他人使用 Eclipse。在极少数情况下,环境差异会导致问题,例如测试在 IDE 中通过但未通过完整构建,或者通过 IDE 签入导致源代码控制系统遭到破坏。不过,一般来说,使用不同的 IDE 不会造成任何问题。有趣的是,随着时间的推移,大多数 Eclipse 用户都转换了。与 IntelliJ 用户配对使他们更喜欢它。

On my current team, some programmers were using IntelliJ IDEA, while others used Eclipse. Environmental differences in rare cases caused issues, such as tests passing in the IDE but not the full build, or check-ins via the IDE causing havoc in the source code control system. Generally, though, use of different IDEs caused no problems. Interestingly, over time most of the Eclipse users switched. Pairing with the IntelliJ users led them to prefer it.

我使用 Eclipse 来处理自动化测试脚本以及研究生产代码的问题。Ruby 插件帮助我们处理 Ruby 和 Watir 脚本,XML 编辑器帮助我们处理 Canoo WebTest 脚本。我们可以运行单元测试并通过 IDE 进行构建。团队中的程序员帮助我设置并开始使用 Eclipse,它节省了大量时间。维护自动化测试要容易得多,而且 IDE 的“同步”视图帮助我记得签入我更改过的所有模块。

I use Eclipse to work with the automated test scripts as well as to research issues with the production code. The Ruby plug-in helps us with our Ruby and Watir scripts, and the XML editor helps with our Canoo WebTest scripts. We can run unit tests and do builds through the IDE. Programmers on the team helped me set up and start using Eclipse, and it has saved huge amounts of time. Maintaining the automated tests is much easier, and the IDE’s “synchronize” view helps me remember to check in all of the modules I’ve changed.

测试工具开始推出它们自己的 IDE 或插件,以与现有的 IDE(如 Eclipse)一起工作。利用这些功能强大、省时、提高质量的工具。

Test tools are starting to come out with their own IDEs or plug-ins to work with existing IDEs such as Eclipse. Take advantage of these powerful, time-saving, quality-promoting tools.

——丽莎

—Lisa

不通过 IDE 进行自动化测试但希望能够查看更改的代码片段的测试人员可以使用 FishEye 等工具,使测试人员能够通过自动构建访问代码。

Testers who aren’t automating tests through an IDE, but who want to be able to look at changed snippets of code, can use tools such as FishEye that enable the testers to get access to the code through the automated build.

在撰写本文时,IDE 已经添加了对动态语言(例如 Ruby、Groovy 和 Python)的支持。使用动态语言的程序员可能更喜欢轻量级的工具,但他们仍然需要支持良好编码实践的好工具,例如 TDD 和重构。

As of this writing, IDEs have added support for dynamic languages such as Ruby, Groovy, and Python. Programmers who use dynamic languages may prefer lighter-weight tools, but they still need good tools that support good coding practices, such as TDD and refactoring.

无论使用何种开发环境和工具,敏捷团队都需要一个框架来集成来自不同程序员的代码更改,运行单元测试以验证没有发生回归错误,并以可部署的格式提供代码。

Regardless of the development environment and tools being used, agile teams need a framework that will integrate code changes from different programmers, run the unit tests to verify no regression bugs have occurred, and provide the code in a deployable format.

构建工具

Build Tools

您的团队需要一些方法来构建软件并创建可部署的 jar、war 或其他类型的文件。这可以使用基于 shell 的工具(例如 make)来完成,但是这些工具有局限性,例如它们工作的平台。我们知道的敏捷团队使用 ant、Nant 和 Maven 等工具来构建他们的项目。这些工具不仅可以管理构建,还可以提供报告和记录构建结果的简便方法,并且可以轻松地与构建自动化和测试工具集成。它们还与 IDE 集成。

Your team needs some way to build the software and create a deployable jar, war, or other type of file. This can be done with shell-based tools such as make, but those tools have limitations, such as the platforms where they work. Agile teams that we know use tools such as ant, Nant, and Maven to build their projects. These tools not only manage the build but also provide easy ways to report and document build results, and they integrate easily with build automation and test tools. They also integrate with IDEs.

构建自动化工具

Build Automation Tools

持续集成是敏捷团队的核心实践。您需要一种方法,不仅可以构建项目,还可以对每个构建运行自动化测试,以确保没有任何问题。每天运行多次的完全自动化且可重现的构建是敏捷团队成功的关键因素。自动化构建工具提供构建结果的电子邮件通知等功能,并且它们与构建和源代码控制工具集成。

Continuous integration is a core practice for agile teams. You need a way to not only build the project but also run automated tests on each build to make sure nothing broke. A fully automated and reproducible build that runs many times a day is a key success factor for agile teams. Automated build tools provide features such as email notification of build results, and they integrate with build and source code control tools.

在撰写本书时常用的工具包括开源工具 CruiseControl、CruiseControl.net、CruiseControl.rb 和 Hudson。发布时可用的其他开源和专有工具包括 AnthillPro、Bamboo、BuildBeat、CI Factory、Team City 和 Pulse,仅举几例。

Commonly used tools as of the writing of this book include the open source tools CruiseControl, CruiseControl.net, CruiseControl.rb, and Hudson. Other open source and proprietary tools available at publication time are AnthillPro, Bamboo, BuildBeat, CI Factory, Team City, and Pulse, just to name a few.

如果没有自动构建过程,您将很难部署用于测试和发布的代码。构建管理和构建自动化工具易于实施,并且对于成功的敏捷项目绝对必要。确保尽早开始构建过程,甚至在开始编码之前。当您发现您需要的功能比当前流程提供的更多时,请尝试使用不同的工具。

Without an automated build process you’ll have a hard time deploying code for testing as well as releasing. Build management and build automation tools are easy to implement and absolutely necessary for successful agile projects. Make sure you get your build process going early, even before you start coding. Experiment with different tools when you find you need more features than your current process provides.

单元测试工具

Unit Test Tools

单元测试工具特定于您编写代码所用的语言。“xUnit”工具通常被敏捷团队使用,并且有许多不同语言的风格,包括用于 Java 的 JUnit、用于 .NET 的 NUnit、用于 Perl 和 Ruby 的 Test::Unit 以及用于 Python 的 PyUnit。

Unit test tools are specific to the language in which you’re coding. “xUnit” tools are commonly used by agile teams, and there’s a flavor for many different languages, including JUnit for Java, NUnit for .NET, Test::Unit for Perl and Ruby, and PyUnit for Python.

行为驱动开发是测试驱动开发的另一种形式,使用 RSpec 和 easyb 等工具阐明预期行为以驱动测试。

Behavior-driven development is another flavor of test-driven development, spelling out expected behavior to drive tests with tools such as RSpec and easyb.

有关行为驱动开发工具的更多信息,请参阅第 9 章支持团队的面向业务的测试工具包”。

See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for more information on behavior-driven development tools.

GUI 代码也可以而且应该以测试先行的方式开发。用于富客户端单元测试的一些工具是 TestNG、Abbot 和 SWTBot。

GUI code can and should be developed test-first as well. Some tools for rich-client unit testing are TestNG, Abbot, and SWTBot.

EasyMock 和 Ruby/Mock 等工具有助于实现模拟对象和测试存根,这是精心设计的单元测试的组成部分。

Tools such as EasyMock and Ruby/Mock help with implementing mock objects and test stubs, an integral part of well-designed unit tests.

请参阅参考书目中的链接和书籍,以帮助您的团队搜索正确的单元测试工具。

See the bibliography for links and books to help your team search for the right unit test tools.

程序员用来编写面向技术的测试的工具也可以用于面向业务的测试。它们是否适合您项目中的该目的取决于您的团队和客户的需求。

The tools programmers use to write technology-facing tests can also be used for business-facing tests. Whether they are suited for that purpose in your project depends on the needs of your team and your customers.

概括

Summary

在本章中,我们解释了支持团队的面向技术的测试的目的,并讨论了哪些团队需要有效地使用它们。

In this chapter, we explained the purpose of technology-facing tests that support the team, and we talked about what teams need to use them effectively.

图像支持编程的面向技术的测试让团队产生最高质量的代码;它们构成了所有其他类型测试的基础。

Technology-facing tests that support programming let the team produce the highest quality code possible; they form the foundation for all other types of testing.

图像这个象限测试的好处包括走得更快和做的更多,但速度和数量永远不应该是最终目标。

The benefits of this quadrant’s tests include going faster and doing more, but speed and quantity should never be the ultimate goal.

图像程序员编写面向技术的测试来支持团队并通过增强系统的内部质量和可测试性为测试人员提供巨大的价值。

Programmers write technology-facing tests that support the team and provide great value to testers by enhancing the internal quality and testability of the system.

图像未能实施与敏捷开发相关的核心实践的团队可能会举步维艰。

Teams that fail to implement the core practices related to agile development are likely to struggle.

图像遗留系统通常是测试驱动开发的最大障碍,但这些问题可以通过增量方法来克服。

Legacy systems usually present the biggest obstacles to test-driven development, but these problems can be overcome with incremental approaches.

图像如果您的团队现在不进行这些测试,您可以通过让其他团队成员参与并获得管理层的支持来帮助他们开始。

If your team doesn’t now do these tests, you can help them get started by engaging other team members and getting support from management.

图像面向技术的测试和支持团队的面向业务的测试之间可能存在一些重叠。然而,当面临选择时,将测试推到最低级别以最大化投资回报率。

There can be some overlap between technology-facing tests and business-facing tests that support the team. However, when faced with a choice, push tests to the lowest level in order to maximize ROI.

图像团队应该建立持续集成、构建和测试流程,以便尽快提供反馈。

Teams should set up continuous integration, build, and test processes in order to provide feedback as quickly as possible.

图像敏捷团队需要用于源代码控制、测试自动化、IDE 和构建管理等任务的工具,以促进支持团队的面向技术的测试。

Agile teams require tools for tasks such as source code control, test automation, IDEs, and build management in order to facilitate technology-facing tests that support the team.

第 8 章支持团队的面向业务的测试

Chapter 8 Business-Facing Tests that Support the Team

图像

在上一章中,我们谈到了程序员测试,即那些帮助程序员确保他们编写的代码正确的低级测试。他们怎么知道应该建造什么?在分阶段和门控方法中,我们尝试通过预先收集需求并尽可能详细地说明来解决这个问题。在使用敏捷实践的项目中,我们完全相信客户理解的故事卡和测试,以帮助编写正确的代码。这些“可理解的”测试是本章的主题。

In the last chapter, we talked about programmer tests, those low-level tests that help programmers make sure they have written the code right. How do they know the right thing to build? In phased and gated methodologies, we try to solve that by gathering requirements up front and putting as much detail in them as possible. In projects using agile practices, we put all our faith in story cards and tests that customers understand in order to help code the right thing. These “understandable” tests are the subject of this chapter.

通过面向业务的测试推动开发

Driving Development with Business-Facing Tests

哎呀,我们正在开始一个迭代,其中的信息不多于适合索引卡的信息,如图8-1所示。

Yikes, we’re starting an iteration with no more information than what fits on an index card, something like what’s shown in Figure 8-1.

图 8-1建立对话的故事

Figure 8-1 Story to set up conversation

图像

信息不多,也不是故意的。故事是对所需功能的简要描述,有助于规划和确定工作的优先级。在传统的瀑布项目中,开发团队可能是给定一个冗长的需求文档,其中包括功能集的每个细节。在敏捷项目中,客户团队和开发团队根据故事展开对话。团队需要某种需求,而且他们需要的级别可以让他们几乎立即开始编写工作代码。为此,我们需要将示例转化为测试,以确认客户真正想要什么。

That’s not much information, and it’s not meant to be. Stories are a brief description of desired functionality and an aid to planning and prioritizing work. On a traditional waterfall project, the development team might be given a wordy requirements document that includes every detail of the feature set. On an agile project, the customer team and development team strike up a conversation based on the story. The team needs requirements of some kind, and they need them at a level that will let them start writing working code almost immediately. To do this, we need examples to turn into tests that will confirm what the customer really wants.

这些面向业务的测试解决了业务需求。这些测试有助于提供全局和足够的细节来指导编码。面向业务的测试基于示例表达需求,并使用客户和开发团队都能理解的语言和格式。示例构成了学习每个功能所需行为的基础,我们使用这些示例作为象限 2 中故事测试的基础(见图8-2)。

These business-facing tests address business requirements. These tests help provide the big picture and enough details to guide coding. Business-facing tests express requirements based on examples and use a language and format that both the customer and development teams can understand. Examples form the basis of learning the desired behavior of each feature, and we use those examples as the basis for our story tests in Quadrants 2 (see Figure 8-2).

图 8-2敏捷测试象限,突出显示象限 2

Figure 8-2 The Agile Testing Quadrants, highlighting Quadrant 2

图像

面向业务的测试也称为“面向客户”、“故事”、“客户”和“验收”测试。“验收测试”一词特别令人困惑,因为它让一些人只想到“用户验收测试”。在敏捷开发的上下文中,验收测试通常指面向业务的测试,但该术语也可以包括来自象限 4 的面向技术的测试,例如客户对系统性能或安全性的标准。在本章中,我们仅讨论通过指导开发和提供快速反馈来支持团队的面向业务的测试。

Business-facing tests are also called “customer-facing,” “story,” “customer,” and “acceptance” tests. The term “acceptance test” is particularly confusing, because it makes some people think only of “user acceptance tests.” In the context of agile development, acceptance tests generally refer to the business-facing tests, but the term could also include the technology-facing tests from Quadrant 4, such as the customer’s criteria for system performance or security. In this chapter, we’re discussing only the business-facing tests that support the team by guiding development and providing quick feedback.

正如我们在前两章中解释的那样,我们呈现这四个象限的顺序与我们可能执行的顺序无关每个象限的活动。象限 2 中面向业务的测试是在编码开始之前为每个故事编写的,因为它们有助于团队了解要编写的代码。与象限 1 中的测试一样,这些测试推动开发,但在更高的层次上。象限 1 的活动确保内部质量,最大限度地提高团队生产力,并最大限度地减少技术债务。象限 2 测试定义和验证外部质量,并帮助我们知道何时完成。

As we explained in the previous two chapters, the order in which we present these four quadrants isn’t related to the order in which we might perform activities from each quadrant. The business-facing tests in Quadrant 2 are written for each story before coding is started, because they help the team understand what code to write. Like the tests in Quadrant 1, these tests drive development, but at a higher level. Quadrant 1 activities ensure internal quality, maximize team productivity, and minimize technical debt. Quadrant 2 tests define and verify external quality, and help us know when we’re done.

第五部分,“生活中的一次迭代”,检查我们在不同象限中执行测试的顺序。

Part V, “An Iteration in the Life,” examines the order in which we perform tests from the different quadrants.

驱动编码的客户测试通常以可执行格式编写,并且是自动化的,因此团队成员可以根据需要经常运行测试,以查看功能是否按预期工作。这些测试或其中的某些子集将成为自动回归套件的一部分,以便未来的开发不会无意中改变系统行为。

The customer tests to drive coding are generally written in an executable format, and automated, so that team members can run the tests as often as they like in order to see if the functionality works as desired. These tests, or some subset of them, will become part of an automated regression suite so that future development doesn’t unintentionally change system behavior.

当我们讨论期望行为的故事和示例时,我们还必须定义非功能性需求,例如性能、安全性和可用性。出色地还要记下手动探索性测试的场景。我们将在有关象限 3 和 4 的章节中讨论这些其他类型的测试活动。

As we discuss the stories and examples of desired behavior, we must also define nonfunctional requirements such as performance, security, and usability. We’ll also make note of scenarios for manual exploratory testing. We’ll talk about these other types of testing activities in the chapters on Quadrants 3 and 4.

我们听到很多关于敏捷团队如何获取需求的问题。我们如何知道我们编写的代码应该做什么?我们如何获得足够的信息来开始编码?如何让客户同声说话,把需求说清楚?我们从哪里开始每个故事?我们如何让客户给我们举个例子?我们如何使用这些来编写故事测试?

We hear lots of questions relating to how agile teams get requirements. How do we know what the code we write should do? How do we obtain enough information to start coding? How do we get the customers to speak with one voice and present their needs clearly? Where do we start on each story? How do we get customers to give us examples? How do we use those to write story tests?

本章解释了我们创建面向业务的测试的策略,这些测试支持团队开发每个故事。让我们从更多地讨论需求开始。

This chapter explains our strategy for creating business-facing tests that support the team as it develops each story. Let’s start by talking more about requirements.

需求困境

The Requirements Quandary

几乎我们所知道的每个开发团队,无论敏捷与否,都在与需求作斗争。传统瀑布项目的团队可能会花费数月的时间来收集需求,结果却发现它们是错误的或很快就过时了。混沌模式下的团队可能根本没有要求,程序员会尽最大可能猜测功能应该如何工作。

Just about every development team we’ve known, agile or not, struggles with requirements. Teams on traditional waterfall projects might invest months in requirements gathering only to have them be wrong or quickly get out of date. Teams in chaos mode might have no requirements at all, with the programmers making their best guess as to how a feature should work.

敏捷开发拥抱变化,但是当需求在迭代过程中发生变化时会发生什么?我们不希望在开始编码之前有很长的需求收集期,但我们如何才能确定我们(和我们的客户)真正了解每个故事的细节?

Agile development embraces change, but what happens when requirements change during an iteration? We don’t want a long requirements-gathering period before we start coding, but how can we be sure we (and our customers) really understand the details of each story?

在敏捷开发中,新特性通常以故事或故事组的形式开始,由客户团队编写。编写故事并不是要弄清楚实现细节,尽管高层讨论会对依赖关系和创建的故事数量产生影响。如果技术团队的某些成员可以参与故事编写会议,这将很有帮助,这样他们就可以输入功能故事并帮助确保将技术故事作为积压工作的一部分。程序员和测试人员还可以帮助客户将故事分解为适当的大小,建议可能更实用的替代方案,并讨论故事之间的依赖关系。

In agile development, new features usually start out life as stories, or groups of stories, written by the customer team. Story writing is not about figuring out implementation details, although high-level discussions can have an impact on dependencies and how many stories are created. It’s helpful if some members of the technical team can participate in story-writing sessions so that they can have input into the functionality stories and help ensure that technical stories are included as part of the backlog. Programmers and testers can also help customers break stories down to appropriate sizes, suggest alternatives that might be more practical to implement, and discuss dependencies between stories.

故事本身并没有提供有关所需功能的太多细节。它们通常只是一个句子,表达了谁想要这个特性,这个特性是什么,以及他们为什么想要它。“作为一名互联网购物者,我需要一种方法来从我的购物车中删除商品,这样我就不必购买不需要的商品”很多想象力。故事仅作为业务专家和开发团队之间持续对话的起点。如果团队成员了解客户试图解决的问题,他们可以提出可能更易于使用和实施的替代方案。

Stories by themselves don’t give much detail about the desired functionality. They’re usually just a sentence that expresses who wants the feature, what the feature is, and why they want it. “As an Internet shopper, I need a way to delete items from my shopping cart so I don’t have to buy unwanted items” leaves a lot to the imagination. Stories are only intended as a starting point for an ongoing dialogue between business experts and the development team. If team members understand what problem the customer is trying to solve, they can suggest alternatives that might be simpler to use and implement.

在客户和开发人员之间的这种对话中,敏捷团队扩展故事,直到他们有足够的信息来编写适当的代码。测试人员帮助引出每个故事的示例和上下文,并帮助客户编写故事测试。这些测试在程序员编写代码时为他们提供指导,并帮助团队了解何时满足了客户的满意度条件。如果您的团队有用例,他们可以帮助补充示例或指导测试以阐明所需的功能(见图8-3)。

In this dialogue between customers and developers, agile teams expand on stories until they have enough information to write appropriate code. Testers help elicit examples and context for each story, and help customers write story tests. These tests guide programmers as they write the code and help the team know when it has met the customers’ conditions of satisfaction. If your team has use cases, they can help to supplement the example or coaching test to clarify the needed functionality (see Figure 8-3).

图 8-3需求的构成

Figure 8-3 The makeup of a requirement

图像

在敏捷开发中,我们承认我们永远无法提前了解故事的所有需求。在故事测试通过的代码完成后,我们还需要做更多的测试,以更好地了解需求和功能应该如何工作。

In agile development, we accept that we’ll never understand all of the requirements for a story ahead of time. After the code that makes the story tests pass is completed, we still need to do more testing to better understand the requirements and how the features should work.

在客户有机会看到团队正在交付什么之后,他们可能会对他们希望它如何工作有不同的想法。客户通常对他们想要什么有一个模糊的概念,并且很难准确定义那是什么。该团队与客户或客户代理合作进行迭代,可能只提供解决方案的核心。团队在多次迭代中不断完善功能,直到定义并交付该功能。

After customers have a chance to see what the team is delivering, they might have different ideas about how they want it to work. Often customers have a vague idea of what they want and a hard time defining exactly what that is. The team works with the customer or customer proxy for an iteration and might deliver just a kernel of a solution. The team keeps refining the functionality over multiple iterations until it has defined and delivered the feature.

能够迭代是敏捷开发提倡小版本和一次开发一小块的原因之一。如果我们的客户对我们在本次迭代中交付的代码的行为不满意,我们可以在他们认为重要的情况下在下一次快速纠正。需求变更几乎是不可避免的。

Being able to iterate is one reason agile development advocates small releases and developing one small chunk at a time. If our customer is unhappy with the behavior of the code we deliver in this iteration, we can quickly rectify that in the next, if they deem it important. Requirements changes are pretty much inevitable.

我们必须尽可能多地了解客户的需求。如果我们的最终用户在我们的位置工作,或者可以前往他们的位置,我们应该与他们坐在一起,与他们一起工作,并尽可能地完成他们的工作。我们不仅会更好地理解他们的需求,而且我们甚至可能会识别出他们没有想到的需求。

We must learn as much as we can about our customers’ wants and needs. If our end users work in our location, or it’s feasible to travel to theirs, we should sit with them, work alongside them, and be able to do their jobs if we can. Not only will we understand their requirements better but we might even identify requirements they didn’t think to state.

测试需要包括比客户规定的要求更多的内容。我们需要测试后期条件、对整个系统的影响以及与其他系统的集成。我们识别风险并根据需要通过测试减轻风险。所有这些因素都指导着我们的编码。

Tests need to include more than the customers’ stated requirements. We need to test for post conditions, impact on the system as a whole, and integration with other systems. We identify risks and mitigate those with tests as needed. All of these factors guide our coding.

通用语言

Common Language

我们还可以使用我们的测试来提供开发团队和业务专家都能理解的通用语言。正如 Brian Marick [2008] 指出的那样,共享语言可以帮助业务人员设想他们想要的功能。它可以帮助程序员精心设计易于扩展的代码。可以表达期望和不期望行为的真实示例,以便业务和技术方面都能理解它们。具有不同背景和观点的人可以访问图片、流程图、电子表格和原型。我们可以使用这些工具来查找示例,然后轻松地将这些示例转化为测试。测试需要以业务用户可以理解的方式编写,但仍可由技术团队执行。

We can also use our tests to provide a common language that’s understood by both the development team and the business experts. As Brian Marick [2008] points out, a shared language helps the business people envision the features they want. It helps the programmers craft well-designed code that’s easy to extend. Real-life examples of desired and undesired behavior can be expressed so that they’re understood by both the business and technical sides. Pictures, flow diagrams, spreadsheets, and prototypes are accessible to people with different backgrounds and viewpoints. We can use these tools to find examples and then easily turn those examples into tests. The tests need to be written in a way that’s comprehensible to a business user reading them yet still executable by the technical team.

面向业务的测试还有助于定义范围,以便每个人都知道什么是故事的一部分,什么不是。许多测试框架现在允许团队创建领域语言并使用该语言定义测试。Fit(集成框架的功能)就是其中之一。

Business-facing tests also help define scope, so that everyone knows what is part of the story and what isn’t. Many of the test frameworks now allow teams to create a domain language and define tests using that language. Fit (Functional for Integrated Framework) is one of those.

可以说我们的客户会向我们提供我们需要的示例,以便我们了解每个故事应该传递的价值。但是,如果他们不知道如何解释他们想要的东西怎么办?在下一节中,我们将建议一些方法来帮助客户定义他们的满意度条件。

It’s fine to say that our customers will provide to us the examples that we need to have in order for us to understand the value that each story should deliver. But what if they don’t know how to explain what they want? In the next section, we’ll suggest ways to help customers define their conditions of satisfaction.

引出需求

Eliciting Requirements

如果您曾经是要求特定软件功能的客户,您就会知道准确表达您想要什么是多么困难。通常,直到您能够看到、感觉到、触摸和使用它时,您才真正确切地知道自己想要什么。我们有很多方法可以帮助我们的客户弄清楚他们想要什么。

If you’ve ever been a customer requesting a particular software feature, you know how hard it is to articulate exactly what you want. Often, you don’t really know exactly what you want until you can see, feel, touch and use it. We have lots of ways to help our customers get clarity about what they want.

问问题

从提问开始。测试人员特别擅长提出各种问题,因为他们了解大局,面向业务的和技术方面的故事,并始终考虑最终用户体验。要问的一般问题的类型是:

Start by asking questions. Testers can be especially good at asking a variety of questions because they are conscious of the big picture, the business-facing and technical aspects of the story, and are always thinking of the end user experience. Types of general questions to ask are:

图像这个故事能解决问题吗?

Is this story solving a problem?

图像如果是这样,我们要解决的问题是什么?

If so, what’s the problem we’re trying to solve?

图像我们能否实施无法解决问题的解决方案?

Could we implement a solution that doesn’t solve the problem?

图像这个故事将如何为企业带来价值?

How will the story bring value to the business?

图像谁是该功能的最终用户?

Who are the end users of the feature?

图像他们将从中获得什么价值?

What value will they get out of it?

图像用户在使用该功能之前和之后会做什么?

What will users do right before and right after they use that feature?

图像我们怎么知道我们已经完成了这个故事?

How do we know we’re done with this story?

丽莎喜欢问的一个问题是,“可能发生的最糟糕的事情是什么?” 最坏的情况往往会产生想法。它们还帮助我们考虑风险并将我们的测试集中在关键区域。另一个好问题是,“可能发生的最好的事情是什么?” 这个问题通常会生成我们的快乐路径测试,但它也可能会揭示一些隐藏的假设。

One question Lisa likes to ask is, “What’s the worst thing that could happen?” Worst-case scenarios tend to generate ideas. They also help us consider risk and focus our tests on critical areas. Another good question is, “What’s the best thing that could happen?” This question usually generates our happy path test, but it might also uncover some hidden assumptions.

使用示例

最重要的是,请客户举例说明该功能应该如何工作。假设这个故事是关于从在线购物车中删除商品的。请客户在白板上画出删除功能的外观。他们是否需要任何额外的功能,例如确认步骤,或者有机会保存该项目以备日后取回?如果无法完成删除,他们希望看到什么?

Most importantly, ask the customer to give you examples of how the feature should work. Let’s say the story is about deleting items out of an online shopping cart. Ask the customer to draw a picture on a whiteboard of how that delete function might look. Do they want any extra features, such as a confirmation step, or a chance to save the item in case they want to retrieve it later? What would they expect to see if the deletion couldn’t be done?

示例可以构成我们测试的基础。我们的挑战是捕获可能用业务领域语言表达的示例,作为可以实际执行的测试。有些客户很乐意使用 Fit 或 FitNesse 等测试工具来表达示例,只要他们可以用他们的领域语言编写示例即可。

Examples can form the basis for our tests. Our challenge is to capture examples, which might be expressed in the business domain language, as tests that can actually be executed. Some customers are comfortable expressing examples using a test tool such as Fit or FitNesse as long as they can write them in their domain language.

让我们用一个简单的故事来探究示例和测试之间的区别(见图8-4)。人们经常混淆这两个术语。

Let’s explore the difference between an example and a test with a simple story (see Figure 8-4). People often get confused between these two terms.

图 8-4用作示例和测试基础的故事

Figure 8-4 Story to use as a base for examples and tests

图像

一个例子看起来像这样:

An example would look something like this:

一页上有 5 个项目。我想选择 20.25 美元的商品 1 并将其放入购物车。我单击下一页,其中还有 5 个项目。我在该页面上以 5.38 美元选择了第二件商品,并将其放入我的购物车。当我说我完成购物时,它会同时显示来自我购物车中第一页和第二页的商品,总计 25.63 美元

There are 5 items on a page. I want to select item 1 for $20.25 and put it in the shopping cart. I click to the next page, which has 5 more items. I select a second item on that page for $5.38 and put it in my shopping cart. When I say I’m done shopping, it will show both the item from the first page and the item from the second page in my shopping cart, with the total of $25.63

测试可能会有很大不同。我们将使用表 8-1中的 Fit 类型格式来向您展示如何表示测试。

The test could be quite a bit different. We’ll use a Fit type format in Table 8-1 to show you how the test could be represented.

表 8-1故事 PA-2 的测试

Table 8-1 Test for Story PA-2

图像

该测试以可执行格式捕获示例。它可能不会使用完全相同的输入,但它封装了示例用户场景。可以编写更多测试用例来测试边界条件、边缘用例和其他场景。

The test captures the example in an executable format. It might not use exactly the same inputs, but it encapsulates the sample user scenario. More test cases can be written to test boundary conditions, edge cases, and other scenarios.

多重观点

每个示例或测试都有一个观点。不同的人会从他们独特的角度写出不同的测试或例子。我们希望尽可能多地捕捉不同的观点,因此请考虑您的用户。

Each example or test has one point of view. Different people will write different tests or examples from their unique perspectives. We’d like to capture as many different viewpoints as we can, so think about your users.

获得正确的需求是许多不同角色的团队成员可以参与进来提供帮助的领域。业务分析师、主题专家、程序员和客户团队的各个成员都可以做出贡献。考虑其他利益相关者,例如您的生产支持团队。他们有非常独特的视角。

Getting the requirements right is an area where team members in many different roles can jump in to help. Business analysts, subject matter experts, programmers, and various members of the customer team all have something to contribute. Think about other stakeholders, such as your production support team. They have a very unique perspective.

我们经常忘记非功能性需求,例如“系统需要运行多长时间?如果失败会怎样?如果我们有传递消息的中间件,我们是否期望消息足够大以至于我们可能需要考虑传输过程中的丢失?或者它们将是一个恒定的大小?如果几个小时没有交通会怎样?系统需要警告人吗?” 对这些类型的需求进行测试通常属于第 3 和第 4 象限,但我们仍然需要编写测试以确保它们完成。

We often forget about nonfunctional requirements such as “How long does the system need to be up? What happens if it fails? If we have middleware that passes messages, do we expect messages to be large enough that we might need to consider loss during transmission? Or will they be a constant size? What happens if there is no traffic for hours? Does the system need to warn someone?” Testing for these types of requirements usually falls into quadrants 3 and 4, but we still need to write tests to make sure they get done.

客户提供给团队的所有例子都很快加起来。我们真的必须将所有这些都变成可执行的测试吗?只要我们有客户告诉我们代码是否按他们想要的方式工作,就不会。使用纸上原型制作等技术,可以在编写一行代码之前测试设计。

All of the examples that customers give to the team add up quickly. Do we really have to turn all of these into executable tests? Not as long as we have the customers there to tell us if the code is working the way they want. With techniques such as paper prototyping, designs can be tested before a line of code is written.

客户团队和开发团队之间密切、持续的协作是获得示例的关键,客户测试基于这些示例驱动编码。沟通是敏捷的核心价值,我们将在下一节详细讨论它。

Close, constant collaboration between the customer team and the developer team is key to obtaining examples on which to base customer tests that drive coding. Communication is a core agile value, and we talk about it more in the next section.

与客户沟通

在理想的世界中,我们的客户每天都可以为我们服务。实际上,许多团队接触到他们的业务专家的机会有限,而且在许多情况下,客户位于不同的地点或时区。尽你所能进行面对面的交谈。当你做不到时,电话会议、电话交谈、电子邮件、即时消息、相机和其他通讯工具将不得不替代。幸运的是,随时可以使用更多工具来促进远程通信。我们听说过一些团队,例如 Weyerhaeuser 的 iLevel 的 Erika Boyer 团队,他们使用可以由远程位置的人员控制的网络摄像头。尽可能靠近以直接对话。

In an ideal world, our customers are available to us all day, every day. In reality, many teams have limited access to their business experts, and in many cases, the customers are in a different location or time zone. Do whatever you can to have face-to face conversations. When you can’t, conference calls, phone conversations, emails, instant messages, cameras, and other communication tools will have to substitute. Fortunately, more tools to facilitate remote communication are available all the time. We’ve heard of teams, such as Erika Boyer’s team at iLevel by Weyerhaeuser, that use webcams that can be controlled by the folks in the remote locations. Get as close to you can to direct conversation.

丽莎的故事

Lisa’s Story

我在一个团队工作,其中的程序员分布在三个时区,而客户则在不同的时区。我们为每次迭代派遣不同的程序员、测试人员和分析师到客户现场,以便每个团队成员至少每三次迭代与客户“面对面”。这在开发人员和客户团队之间建立了信任和信心。其余时间我们都是通过电话、公开电话会议、即时消息等方式提问。通过基于回顾性讨论的持续微调,我们成功地让客户满意甚至高兴。

I worked on a team where the programmers were spread through three time zones and the customers were in a different one. We sent different programmers, testers, and analysts to the customer site for every iteration, so that each team member had “face time” with the customers at least every third iteration. This built trust and confidence between the developer and customer teams. The rest of the time we used phone calls, open conference calls, and instant messages to ask questions. With continual fine-tuning based on retrospective discussions, we succeeded in satisfying and even delighting the customers.

——丽莎

—Lisa

即使有客户并且通信线路畅通无阻,也需要管理通信。我们想与客户团队的每一位成员交谈,但他们都有不同的观点。如果我们对某项功能的工作方式有多个不同的版本,我们将不知道要编写什么代码。让我们考虑让客户就每个故事的满意度条件达成一致的方法。

Even when customers are available and lines of communication are wide open, communication needs to be managed. We want to talk to each member of the customer team, but they all have different viewpoints. If we get several different versions of how a piece of functionality should work, we won’t know what to code. Let’s consider ways to get customers to agree on the conditions of satisfaction for each story.

提前澄清

Advance Clarity

如果您的客户团队由来自组织不同部门的人员组成,那么他们之间可能对特定故事的确切意图存在冲突意见。在丽莎的公司,业务发展要产生收入的功能,运营需要减少电话支持电话的功能,财务需要简化会计、现金管理和报告的功能。令人惊讶的是,不同观点的人可以对同一个故事做出如此多的独特解释。

If your customer team consists of people from different parts of the organization, there may be conflicting opinions among them about exactly what’s intended by a particular story. In Lisa’s company, business development wants features that generate revenue, operations wants features that cut down on phone support calls, and finance wants features that streamline accounting, cash management, and reporting. It’s amazing how many unique interpretations of the same story can emerge from people who have differing viewpoints.

丽莎的故事

Lisa’s Story

虽然我们在最初实施 Scrum 时有一个产品负责人,但我们仍然从不同的客户那里得到不同的指令。管理层决定任命一位具有广泛领域和运营知识的副总裁作为新的产品负责人。他负责让所有利益相关者预先就每个故事的含义达成一致。他和客户团队的其他成员定期会面,讨论即将到来的主题和故事,并就优先事项和满意条件达成一致。他称之为“预先清晰”。

Although we had a product owner when we first implemented Scrum, we still got different directives from different customers. Management decided to appoint a vice president with extensive domain and operations knowledge as the new product owner. He is charged with getting all of the stakeholders to agree on each story’s implications up front. He and the rest of the customer team meet regularly to discuss upcoming themes and stories, and to agree on priorities and conditions of satisfaction. He calls this “advance clarity.”

——丽莎

—Lisa

产品负责人是 Scrum 中的一个角色。他不仅负责实现预先的清晰度,而且还负责在确定故事优先级时充当“客户代表”。不过有一个缺点。当您通过一个人汇集许多不同观点的需求时,可能会丢失一些东西。理想情况下,开发团队应该与客户团队坐在一起,学习如何完成客户的工作。如果我们充分了解客户的需求以执行其日常任务,我们就有更好的机会开发出能够正确支持这些任务的软件。

A Product Owner is a role in Scrum. He’s responsible not only for achieving advance clarity but also for acting as the “customer representative” in prioritizing stories. There’s a downside, though. When you funnel the needs of many different viewpoints through one person, something can be lost. Ideally, the development team should sit together with the customer team and learn how to do the customer’s work. If we understand the customer’s needs well enough to perform its daily tasks, we have a much better chance of producing software that properly supports those tasks.

珍妮特的故事

Janet’s Story

我们的团队一开始没有实施产品负责人的角色,而是使用团队中的领域专家来确定优先级和清晰度。它运作良好,但达成共识需要召开多次会议,因为每个人都有不同的经历。该产品更适合它,但也需要权衡取舍。许多会议意味着领域专家并不总是可以回答程序员的问题,因此编码速度比预期的要慢。

Our team didn’t implement the product owner role at first and used the domain experts on the team to determine prioritization and clarity. It worked well, but the achieving consensus took many meetings because each person had different experiences. The product was better for it, but there were trade-offs. The many meetings meant the domain experts were not always available for answering questions from the programmers, so coding was slower than anticipated.

有四个独立的项目团队致力于同一产品,但每个团队都专注于不同的功能。经过几次回顾和大量的问题解决会议后,每个项目团队都任命了一名产品负责人。会议次数大幅减少,因为大多数业务决策都是由领域专家针对他们的特定项目做出的。如果有任何意见分歧,将为所有领域专家举行会议,并且产品负责人会协助就某个问题达成共识。决策速度更快,领域专家更容易回答团队的问题,并且能够跟上验收测试。

There were four separate project teams working on the same product, but each one was focused on different features. After several retrospectives, and a lot of problem-solving sessions, each project team appointed a Product Owner. The number of meetings was cut down significantly because most business decisions were made by the domain experts on their particular project. Meetings were held for all of the domain experts if there were any differences of opinion, and the Product Owner facilitated bringing consensus on an issue. Decisions were made much faster, the domain experts were more available for answering questions by the team, and were able to keep up with the acceptance tests.

——珍妮特

—Janet

无论您的团队选择汇集不同的观点,重要的是只有“客户的一个声音”呈现给团队。

However your team chooses to bring together varying viewpoints, it is important that there is only “one voice of the customer” presented to the team.

我们说产品所有者提供满意的条件。让我们更仔细地看看我们的意思。

We said that product owners provide conditions of satisfaction. Let’s look more closely at what we mean.

满意的条件

Conditions of Satisfaction

整个发行版以及每个功能或故事都有令人满意的条件。验收测试有助于定义故事验收。除非预先同意故事的满足条件,否则您的开发团队无法成功交付业务所需的内容。客户团队需要“用同一个声音说话”。如果您从不同的利益相关者那里得到不同的要求,您可能需要推迟并推迟故事,直到您有一个明确的业务满意度条件列表。请客户代表提供每个故事的最少信息量,以便您可以通过富有成效的对话开始每次迭代。

There are conditions of satisfaction for the whole release as well as for each feature or story. Acceptance tests help define the story acceptance. Your development team can’t successfully deliver what the business wants unless conditions of satisfaction for a story are agreed to up front. The customer team needs to “speak with one voice.” If you’re getting different requirements from different stakeholders, you might need to push back and put off the story until you have a firm list of business satisfaction conditions. Ask the customer representative to provide a minimum amount of information on each story so that you can start every iteration with a productive conversation.

了解客户团队要求的最好方法是与客户面对面交谈。因为每个人都在与“需求”作斗争,所以有一些工具可以帮助客户团队完成每个故事。满意的条件不仅应包括故事所传达的特征,还应包括对更大系统的影响。

The best way to understand the customer team’s requirements is to talk with the customers face to face. Because everyone struggles with “requirements,” there are tools to help the customer team work through each story. Conditions of satisfaction should include not only the features that the story delivers but also the impacts on the larger system.

Lisa 的产品负责人使用清单格式来解决以下问题:

Lisa’s product owner uses a checklist format to sort out issues such as:

图像业务满意条件

Business satisfaction conditions

图像对网站、文档、发票、表格或报告等现有功能的影响

Impact on existing functions such as the website, documents, invoices, forms, or reports

图像法律考虑

Legal considerations

图像对定期调度流程的影响

The impact on regularly scheduled processes

图像对 UI 故事模型的引用

References to mock-ups for UI stories

图像帮助文本,或由谁提供

Help text, or who will provide it

图像测试用例

Test cases

图像数据迁移(视情况而定)

Data migration (as appropriate)

图像需要发生的内部沟通

Internal communication that needs to happen

图像与业务合作伙伴和供应商的外部沟通

External communication to business partners and vendors

产品所有者使用模板将此信息放在团队的 wiki 上,以便团队成员了解故事和开始编写测试时可以使用它。

The product owner uses a template to put this information on the team’s wiki so that it can be used as team members learn about the stories and start writing tests.

第 9 章,“支持团队的面向业务的测试工具包”包括示例检查表以及其他用于表达需求的工具。

Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” includes example checklists as well as other tools for expressing requirements.

这些条件基于客户团队为故事所做的关键假设和决策。他们通常来自与客户就每个故事的高级验收标准进行的对话。讨论满意度条件有助于识别有风险的假设,并增加团队编写和正确估计完成故事所需的所有任务的信心。

These conditions are based on key assumptions and decisions made by the customer team for a story. They generally come out of conversations with the customer about high-level acceptance criteria for each story. Discussing conditions of satisfaction helps identify risky assumptions and increases the team’s confidence in writing and correctly estimating all of the tasks that are needed to complete the story.

涟漪效应

Ripple Effects

在敏捷开发中,我们一次只关注一个故事。每个故事通常是整个应用程序的一个小组成部分,但它可能会产生很大的连锁反应。一个新故事就像一块小石头掉入应用程序的水中,我们可能不会考虑由此产生的波浪会遇到什么。当我们在每次迭代中专注于少数故事时,很容易忘记大局。

In agile development, we focus on one story at a time. Each story is usually a small component of the overall application, but it might have a big ripple effect. A new story drops like a little stone into the application water, and we might not think about what the resulting waves might run into. It’s easy to lose track of the big picture when we’re focusing on a small number of stories in each iteration.

Lisa 的团队发现列出系统中可能受故事影响的所有部分很有帮助。团队可以检查每个“测试点”以查看它可能生成的需求和测试用例。一个小而无辜的故事可能会产生广泛的影响,它所涉及的应用程序的每个部分都可能呈现出另一个层次的复杂性。您需要了解任何代码更改的所有潜在影响。列一个清单是一个很好的起点。在迭代的前几天,团队可以进一步研究和分析受影响的区域,看看是否需要更多的任务卡来覆盖所有这些区域。

Lisa’s team finds it helpful to make a list of all of the parts of the system that might be affected by a story. The team can check each “test point” to see what requirements and test cases it might generate. A small and innocent story might have a wide-ranging impact, and each part of the application that it touches might present another level of complexity. You need to be aware of all the potential impacts of any code change. Making a list is a good place to start. In the first few days of the iteration, the team can research and analyze affected areas further and see whether any more task cards are needed to cover them all.

珍妮特的故事

Janet’s Story

在我参与的一个项目中,我们使用了一个简单的电子表格,其中列出了被测应用程序的所有高级功能。在发布计划期间,以及在每次新迭代开始时,我们审查了列表并考虑新的或更改的功能将如何影响这些领域。这成为确定每个功能区域需要进行何种级别的测试的起点。这种影响分析是对实际故事测试的补充,使我们的团队能够看到全局以及更改对系统其余部分的影响。

In one project I was on, we used a simple spreadsheet that listed all of the high-level functionality of the application under test. During release planning, and at the start of each new iteration, we reviewed the list and thought about how the new or changing functionality would affect those areas. That became the starting point for determining what level of testing needed to be done in each functional area. This impact analysis was in addition to the actual story testing and enabled our team to see the big picture and the impact of the changes to the rest of the system.

——珍妮特

—Janet

看起来很小但会影响系统意外区域的故事可能会反过来咬你一口。如果您的团队忘记考虑所有依赖关系,并且如果新代码与现有功能相交,您的故事可能需要比计划更长的时间才能完成。确保您的故事测试包括实施新功能时不太明显的后果。

Stories that look small but that impact unexpected areas of the system can come back to bite you. If your team forgets to consider all dependencies, and if the new code intersects with existing functionality, your story might take much longer than planned to finish. Make sure your story tests include the less obvious fallout from implementing the new functionality.

第 16 章开始运行”和第 17 章迭代启动”举例说明团队何时以及如何计划客户测试并探索每个故事的更广泛影响。

Chapter 16, “Hit the Ground Running,” and Chapter 17, “Iteration Kickoff,” give examples of when and how teams can plan customer tests and explore the wider impact of each story.

花时间确定每个故事提供的核心价值,并找出一种渐进的方法来开发它。计划编写测试、编写代码和测试代码的小增量。这样,您的象限 2 测试可确保您按计划交付最小值。

Take time to identify the central value each story provides and figure out an incremental approach to developing it. Plan small increments of writing tests, writing code, and testing the code some more. This way, your Quadrant 2 tests ensure you’ll deliver the minimum value as planned.

薄片,小块

Thin Slices, Small Chunks

写故事是一件棘手的事情。开发团队在估算新故事时,可能会发现一些故事太大,因此会要求客户团队返回将它们分解成更小的故事。故事也可能太小,可能需要与其他故事结合或简单地视为任务。敏捷开发(包括测试)一次只处理一小部分功能。

Writing stories is a tricky business. When the development team estimates new stories, it might find some stories too big, so it will ask the customer team to go back and break them into smaller stories. Stories can be too small as well, and might need to be combined with others or simply treated as tasks. Agile development, including testing, takes on one small chunk of functionality at a time.

当你的团队开始一个新的项目或主题时,请产品负责人在该主题的第一次迭代之前将所有相关故事带到头脑风暴会议中。让产品负责人和其他感兴趣的利益相关者解释这些故事。您可能会发现某些故事需要细分,或者需要编写额外的故事来填补空白。

When your team embarks on a new project or theme, ask the product owner to bring all of the related stories to a brainstorming session prior to the first iteration for that theme. Have the product owner and other interested stakeholders explain the stories. You might find that some stories need to be subdivided or that additional stories need to be written to fill in gaps.

在您了解每个故事应该传递什么价值以及它如何适应系统的上下文之后,您可以将故事分解成小的、可管理的部分。您可以编写客户测试来定义这些小增量,同时牢记对较大应用程序的影响。

After you understand what value each story should deliver and how it fits in the context of the system, you can break the stories down into small, manageable pieces. You can write customer tests to define those small increments, while keeping in mind the impact on the larger application.

编写指导开发的客户测试的一种聪明的增量方法是从“薄片”开始,它遵循从一端到另一端的快乐路径。识别薄片,也称为“钢丝”或“示踪子弹”,可以在主题级别完成,用于验证整体架构。这条钢丝将所有的部件连接在一起,在它牢固之后,可以添加更多的功能。

A smart incremental approach to writing customer tests that guide development is to start with the “thin slice” that follows a happy path from one end to the other. Identifying a thin slice, also called a “steel thread” or “tracer bullet,” can be done on a theme level, where it’s used to verify the overall architecture. This steel thread connects all of the components together, and after it’s solid, more functionality can be added.

有关探索性测试的更多信息,请参阅第 10 章,“评价产品的面向业务的测试”。

See Chapter 10, “Business-Facing Tests that Critique the Product,” for more about exploratory testing.

我们发现这种策略也适用于故事层面。越早构建端到端路径,就能越早进行有意义的测试、获得反馈、开始自动化测试和探索性测试。从一小部分可以测试的最精简的功能开始。这可以被认为是关键路径。对于用户界面,这可能从简单地从一页导航到下一页开始。我们可以将其展示给客户,看看流程是否有意义。我们可以编写一个简单的自动化 GUI 测试。对于本章开头的免运费门槛故事,我们可以从验证用于汇总订单总额的逻辑开始,并确定它是否符合免费送货条件,无需担心它在 UI 上的外观。我们可以使用 FitNesse 等功能测试工具对其进行自动化测试。

We find this strategy works at the story level, too. The sooner you can build the end-to-end path, the sooner you can do meaningful testing, get feedback, start automating tests, and start exploratory testing. Begin with a thin slice of the most stripped-down functionality that can be tested. This can be thought of as the critical path. For a user interface, this might start with simply navigating from one page to the next. We can show this to the customer and see whether the flow makes sense. We could write a simple automated GUI test. For the free-shipping threshold story at the beginning of this chapter, we might start by verifying the logic used to sum up the order total and determine whether it qualifies for free shipping, without worrying about how it will look on the UI. We could automate tests for it with a functional test tool such as FitNesse.

有关回归测试自动化的更多信息,请参见第 IV 部分,“自动化”。

See Part IV, “Automation,” for more about regression test automation.

在薄片工作后,我们可以为下一个块或功能层编写客户测试,并编写使这些测试通过的代码。现在我们也将收到针对这一小增量的反馈。也许我们添加 UI 来显示结账页面,显示该订单符合免费送货条件,或者添加层来持久更新数据库。我们可以添加到我们为第一遍编写的自动化测试中。这是一个“编写测试——编写代码——运行测试——学习”的过程。如果您这样做,您就会知道您的团队生成的所有代码都能让客户满意并且在每个阶段都能正常工作。

After the thin slice is working, we can write customer tests for the next chunk or layer of functionality, and write the code that makes those tests pass. Now we’ll have feedback for this small increment, too. Maybe we add the UI to display the checkout page showing that the order qualified for free shipping, or add the layer to persist updates to the database. We can add on to the automated tests we wrote for the first pass. It’s a process of “write tests—write code—run tests—learn.” If you do this, you know that all of the code your team produces satisfies the customer and works properly at each stage.

丽莎的故事

Lisa’s Story

我的团队发现我们必须专注于完成一个简单的薄片并以微小的增量添加到它。在我们这样做之前,我们往往会卡在故事的一部分。例如,如果我们有一个包含四个屏幕的 UI 流,我们会在第一个屏幕上如此投入,以至于我们可能无法到达最后一个屏幕,并且没有有效的端到端路径。通过从端到端的快乐路径开始并一次添加功能,我们可以确保交付所需的最低价值。

My team has found that we have to focus on accomplishing a simple thin slice and add to it in tiny increments. Before we did this, we tended to get stuck on one part of the story. For example, if we had a UI flow that included four screens, we’d get so involved in the first one that we might not get to the last one, and there was no working end-to-end path. By starting with an end-to-end happy path and adding functionality a step at a time, we can be sure of delivering the minimum value needed.

这是我们流程的一个例子。这个故事是在建立公司退休计划的过程中添加一个新的有条件的步骤。此步骤允许用户选择共同基金投资组合,但并非每个用户都可以使用此功能。退休计划建立功能是用旧的、设计不佳的遗留代码编写的。我们计划在新架构中编写新页面,但将新旧代码链接在一起很棘手且容易出错。我们将故事分解成多个片段,这些片段看起来可能很小,但这使我们能够管理风险并最大限度地减少编码和测试故事所需的时间。图 8-5显示了为这个故事计划的增量步骤图。

Here’s an example of our process. The story was to add a new conditional step to the process of establishing a company’s retirement plan. This step allows users to select mutual fund portfolios, but not every user has access to this feature. The retirement plan establishment functionality is written in old, poorly designed legacy code. We planned to write the new page in the new architecture, but linking the new and old code together is tricky and error prone. We broke the story down into slices that might look tiny but that allowed us to manage risk and minimize the time needed to code and test the story. Figure 8-5 shows a diagram of incremental steps planned for this story.

图 8-5增量步骤

Figure 8-5 Incremental steps

图像

#1 thin slice 是根据属性插入一个新的空页面。虽然这对我们的客户来说并不多,但它让我们可以测试新旧代码之间的桥梁,然后验证计划建立导航是否仍然正常工作。第 2 部分介绍了一些业务逻辑:如果公司没有可用的共同基金投资组合,请跳到我们尚未更改的基金选择步骤。如果有可用的基金投资组合,则在新的步骤 3 中显示它们。在切片 #3 中,我们更改基金选择步骤,添加逻辑以显示构成投资组合的基金。切片 #4 在建立过程的各个步骤之间添加导航元素。

The #1 thin slice is to insert a new, empty page based on a property. While it’s not much for our customers to look at, it lets us test the bridge between old and new code, and then verify that the plan establishment navigation still works properly. Slice #2 introduces some business logic: If no mutual fund portfolios are available for the company, skip to the fund selection step, which we’re not changing yet. If there are fund portfolios available, display them on the new step 3. In slice #3, we change the fund selection step, adding logic to display the funds that make up the portfolios. Slice #4 adds navigational elements between various steps in the establishment process.

我们编写了客户测试来定义每个切片。当程序员完成每一项时,我们对其进行手动测试并将其展示给我们的客户。发现的任何问题都会立即得到解决。我们为 slice #1 编写了一个自动化的 GUI 测试,并在完成剩余步骤后添加到其中。这个故事很困难,因为旧的遗留代码与新架构交互,但逐步的方法使实施顺利,并节省了时间。

We wrote customer tests to define each slice. As the programmers completed each one, we manually tested it and showed it to our customers. Any problems found were fixed immediately. We wrote an automated GUI test for slice #1, and added to it as the remaining steps were finished. The story was difficult because of the old legacy code interacting with the new architecture, but the stepwise approach made implementation smooth, and saved time.

当我们绘制这样的图表以将故事分成多个部分时,我们将它们的照片上传到我们的团队 wiki,以便我们的远程团队成员也可以看到它们。每一步完成后,我们都会将其勾选以提供即时视觉反馈。

When we draw diagrams such as this to break stories into slices, we upload photos of them to our team wiki so our remote team member can see them too. As each step is finished, we check it off in order to provide instant visual feedback.

——丽莎

—Lisa

如果为故事编写客户测试的任务看起来令人困惑或难以应付,您的团队可能需要将故事分解成更小的步骤或块。一次一小步地完成故事有助于分散测试工作,这样它就不会被推到迭代的末尾。它还可以让您更好地了解自己的进度,并帮助您知道何时完成——我们将在下一节探讨这个主题。

If the task of writing customer tests for a story seems confusing or overwhelming, your team might need to break the story into smaller steps or chunks. Finishing stories a small step at a time helps spread out the testing effort so that it doesn’t get pushed to the end of the iteration. It also gives you a better picture of your progress and helps you know when you’re done—a subject we’ll explore in the next section.

查看 Gerard Meszaros 的文章“Using Storyotypes to Split Bloated XP Stories”的参考书目。

Check the bibliography for Gerard Meszaros’s article “Using Storyotypes to Split Bloated XP Stories.”

我们怎么知道我们已经完成了?

How Do We Know We’re Done?

我们有支持团队的面向业务的测试——这些测试是为确保满足条件而编写的。他们从快乐的道路开始,并表明故事符合预期需要。它们涵盖了各种用户场景,并确保系统的其他部分不会受到不利影响。这些测试已经运行,并且通过了(或者至少他们已经确定了要修复的问题)。

We have our business-facing tests that support the team—those tests that have been written to ensure the conditions of satisfaction have been met. They start with the happy path and show that the story meets the intended need. They cover various user scenarios and ensure that other parts of the system aren’t adversely affected. These tests have been run, and they pass (or at least they’ve identified issues to be fixed).

我们现在完成了吗?我们可能是,但我们还不确定。真正的考验是软件的用户是否可以执行故事应该提供的操作。象限 3 和 4 的活动,例如探索性测试、可用性测试和性能测试,将帮助我们找到答案。现在,我们只需要做一些客户测试,以确保我们已经捕获了所有需求。业务用户或产品负责人是确定每个需求是否已交付的合适人选,因此他们是现阶段进行探索的合适人选。

Are we done now? We could be, but we’re not sure yet. The true test is whether the software’s user can perform the action the story was supposed to provide. Activities from Quadrants 3 and 4, such as exploratory testing, usability testing, and performance testing will help us find out. For now, we just need to do some customer tests to ensure that we have captured all of the requirements. The business users or product owners are the right people to determine whether every requirement has been delivered, so they’re the right people to do the exploring at this stage.

当所有测试都通过并且确定了任何遗漏的需求时,我们就完成了支持程序员寻求执行“正确事情”的代码的目的。这并不意味着我们完成了测试。我们将在接下来的章节中详细讨论这一点。

When the tests all pass and any missed requirements have been identified, we are done for the purpose of supporting the programmers in their quest for code that does the “right thing.” It does not mean we are done testing. We’ll talk much more about that in the chapters that follow.

客户测试的另一个目标是识别高风险区域并确保编写代码以巩固这些区域。风险管理是任何软件开发方法中的基本实践,测试人员在识别和减轻风险方面发挥作用。

Another goal of customer tests is to identify high-risk areas and make sure the code is written to solidify those. Risk management is an essential practice in any software development methodology, and testers play a role in identifying and mitigating risks.

测试降低风险

Tests Mitigate Risk

编写客户测试不仅是为了定义代码的预期行为,也是为了管理风险。通过测试推动开发并不意味着我们会预先确定每一个需求或能够在我们完成时完美地预测。它确实让我们有机会识别风险并通过可执行的测试用例来减轻风险。风险分析并不是一项新技术。敏捷开发通过将业务价值优先化为小的、经过测试的可交付部分并让客户参与增量验收,从而从本质上减轻了一些风险。然而,我们仍然应该集思广益了解潜在事件、它们可能发生的可能性以及如果它们确实发生对组织的影响,以便可以采用正确的缓解策略。

Customer tests are written not only to define expected behavior of the code but to manage risk. Driving development with tests doesn’t mean we’ll identify every single requirement up front or be able to predict perfectly when we’re done. It does give us a chance to identify risks and mitigate them with executable test cases. Risk analysis isn’t a new technique. Agile development inherently mitigates some risks by prioritizing business value into small, tested deliverable pieces and by having customer involvement in incremental acceptance. However, we should still brainstorm potential events, the probability they might occur, and the impact on the organization if they do happen so that the right mitigation strategy can be employed.

如果测试是针对不太可能的边缘情况,则对预定义测试进行编码效果不佳。虽然我们不想只测试快乐路径,但它是一个很好的起点。知道快乐路径后,我们可以定义风险最高的场景——不仅有坏结果而且很有可能发生的情况。

Coding to predefined tests doesn’t work well if the tests are for improbable edge cases. While we don’t want to test only the happy path, it’s a good place to start. After the happy path is known, we can define the highest risk scenarios—cases that not only have a bad outcome but also have a good possibility of happening.

除了向客户团队提出诸如“可能发生的最糟糕的事情是什么?”之类的问题之外,还要向程序员提出如下问题:“这部分代码的后置条件是什么?什么应该保存在数据库中?我们应该寻找什么样的行为?” 指定测试以涵盖操作的潜在风险结果。

In addition to asking the customer team questions such as “What’s the worst thing that could happen?,” ask the programmers questions like these: “What are the post conditions of this section of code? What should be persisted in the database? What behavior should we look for down the line?” Specify tests to cover potentially risky outcomes of an action.

丽莎的故事

Lisa’s Story

我的团队会考虑最坏的情况,以帮助我们确定客户测试。例如,我们计划了一个故事,用几个新选项重写多步骤帐户创建向导的第一步。我们问自己如下问题:“当用户提交第一页时,数据库中插入了什么数据?是否触发了任何其他更新?我们是否需要对整个帐户设置过程进行回归测试?用户帐户在设置后可能进行的活动呢?” 我们可能需要测试帐户的整个生命周期。我们没有时间进行不必要的测试,因此关于测试什么的决定至关重要。正确的测试可以帮助我们减轻变更带来的风险。

My team considers worst-case scenarios in order to help us identify customer tests. For example, we planned a story to rewrite the first step of a multistep account creation wizard with a couple of new options. We asked ourselves questions such as the following: “When the user submits that first page, what data is inserted in the database? Are any other updates triggered? Do we need to regression test the entire account setup process? What about activities the user account might do after setup?” We might need to test the entire life cycle of the account. We don’t have time to test more than necessary, so decisions about what to test are critical. The right tests help us mitigate the risk brought by the change.

——丽莎

—Lisa

程序员可以识别代码的脆弱部分。这个故事是否涉及将遗留代码与新架构拼接在一起?被更改的代码是否与另一个系统交互或依赖于第三方软件?通过与程序员和其他团队成员讨论潜在影响和风险区域,我们可以计划适当的测试活动。

Programmers can identify fragile parts of the code. Does the story involve stitching together legacy code with a new architecture? Does the code being changed interact with another system or depend on third-party software? By discussing potential impacts and risky areas with programmers and other team members, we can plan appropriate testing activities.

还有一个风险。我们可能会提前编写详细的测试用例,以至于团队在树中迷失了森林;也就是说,当我们专注于可能被证明无关紧要的细节时,我们可能会忘记大局。

There’s another risk. We might get so involved writing detailed test cases up front that the team loses the forest in the trees; that is, we can forget the big picture while we concentrate on details that might prove irrelevant.

危险:忘记大局

Peril: Forgetting the Big Picture

很容易养成只测试单个故事或将测试建立在程序员告诉您的代码内容的习惯上。如果您发现自己在发布后期发现故事之间存在集成问题,或者在故事“完成”后发现大量需求缺失,请采取措施减轻这种风险。

It’s easy to slip into the habit of testing only individual stories or basing your testing on what the programmer tells you about the code. If you find yourself finding integration problems between stories late in the release or that a lot of requirements are missing after the story is “done,” take steps to mitigate this peril.

始终考虑每个单独的故事如何影响系统的其他部分。使用真实的测试数据,使用具体的例子作为测试的基础,并进行大量的白板讨论(或虚拟的等价物)以确保每个人都理解这个故事。确保程序员在编写任何测试之前不开始编码,并使用探索性测试来查找故事之间的差距。

Always consider how each individual story impacts other parts of the system. Use realistic test data, use concrete examples as the basis of your tests, and have a lot of whiteboard discussions (or their virtual equivalent) in order to make sure everyone understands the story. Make sure the programmers don’t start coding before any tests are written, and use exploratory testing to find gaps between stories.

记住最终目标和大局。

Remember the end goal and the big picture.

作为一个敏捷团队,我们在短迭代中工作,因此在我们开始之前对花在编写测试上的时间进行时间限制很重要。每次迭代完成后,花时间评估预先提供更多细节是否有帮助。是否有足够的测试来让团队走上正轨?是否因为故事被误解而浪费了很多时间?Lisa 的团队发现最好在编码之前编写高级故事测试,在编码开始后编写详细的测试用例,然后在代码交付时对其进行探索性测试,以便为团队提供更多信息并帮助进行必要的调整.

As an agile team, we work in short iterations, so it’s important to time-box the time spent writing tests before we start. After each iteration is completed, take the time to evaluate whether more detail up front would have helped. Were there enough tests to keep the team on track? Was there a lot of wasted time because the story was misunderstood? Lisa’s team has found it best to write high-level story tests before coding, to write detailed test cases once coding starts, and then to do exploratory testing on the code as it’s delivered in order to give the team more information and help make needed adjustments.

珍妮特从事的项目有一些非常密集的计算。为了确保计算正确完成,在开始编码之前创建详细示例和测试所花费的时间是值得的。了解领域和每个故事的影响对于评估风险和选择正确的缓解策略至关重要。

Janet worked on a project that had some very intensive calculations. The time spent creating detailed examples and tests before coding started, in order to ensure that the calculations were done correctly, was time well spent. Understanding the domain, and the impact of each story, is critical to assessing the risk and choosing the correct mitigation strategy.

虽然面向业务的测试可以帮助降低风险,但其他类型的测试也很重要。例如,许多最严重的问题通常是在手动探索性测试期间发现的。性能、安全性、稳定性和可用性也是风险的来源。在第 3 和第 4 象限的章节中讨论了减轻这些其他风险的测试。

While business-facing tests can help mitigate risks, other types of tests are also critical. For example, many of the most serious issues are usually uncovered during manual exploratory testing. Performance, security, stability, and usability are also sources of risk. Tests to mitigate these other risks are discussed in the chapters on Quadrants 3 and 4.

试验并找到您的团队可以在使用前期细节和保持专注于大局之间取得平衡的方法。短期敏捷迭代的美妙之处在于,您经常有机会评估您的流程的运作方式,以便您可以进行持续改进。

Experiment and find ways that your team can balance using up-front detail and keeping focused on the big picture. The beauty of short agile iterations is that you have frequent opportunities to evaluate how your process is working so that you can make continual improvements.

可测试性和自动化

Testability and Automation

当敏捷团队中的程序员准备好进行测试驱动开发时,他们会针对故事使用面向业务的测试,以便知道要编写什么代码。从测试开始意味着每个人都在思考设计代码的最佳方式,以使测试更容易。象限 2 中面向业务的测试表示为自动化测试。它们需要清晰易懂、易于运行并提供快速反馈;否则,它们不会被使用。

When programmers on an agile team get ready to do test-driven development, they use the business-facing tests for the story in order to know what to code. Working from tests means that everyone thinks about the best way to design the code to make testing easier. The business-facing tests in Quadrant 2 are expressed as automated tests. They need to be clearly understood, easy to run, and provide quick feedback; otherwise, they won’t get used.

可以为程序员编写手动测试脚本,让他们在签入代码之前执行,这样他们就可以确保满足客户的条件,但期望他们会长期如此麻烦是不现实的。当必须每两周或每 30 天交付一次有意义的商业价值时,信息必须是直接和自动的。缺乏经验的敏捷团队可能会接受在开发人员测试级别比在客户测试级别更容易通过自动化测试驱动编码的需要等级。然而,如果没有客户测试,程序员就很难知道要编写哪些单元测试。

It’s possible to write manual test scripts for the programmers to execute before they check in code so that they can make sure they satisfied the customer’s conditions, but it’s not realistic to expect they’ll go to that much trouble for long. When meaningful business value has to be delivered every two weeks or every 30 days, information has to be direct and automatic. Inexperienced agile teams might accept the need to drive coding with automated tests at the developer test level more easily than at the customer test level. However, without the customer tests, the programmers have a much harder time knowing what unit tests to write.

每个敏捷团队都必须找到一个流程来编写和自动化面向业务的测试,以推动开发。只对面向技术的测试进行自动化的团队发现,他们可以拥有没有错误的代码,但这些代码并不能满足客户的需求。不自动执行任何测试的团队将背负技术债务。

Each agile team must find a process of writing and automating business-facing tests that drive development. Teams that automate only technology-facing tests find that they can have bug-free code that doesn’t do what the customer wants. Teams that don’t automate any tests will anchor themselves with technical debt.

第 IV 部分测试自动化”将指导您制定自动化策略。

Part IV, “Test Automation,” will guide you as you develop an automation strategy.

象限 2 包含许多不同类型的测试和活动。我们需要合适的工具来促进收集、讨论和交流示例和测试。如果团队位于同一地点,则简单的工具(如纸张或白板)可以很好地收集示例。更复杂的工具可帮助团队编写面向业务的测试,以可执行、可自动化的格式指导开发。在下一章中,我们将了解引出示例以及编写、交流和执行支持团队的面向业务的测试所需的各种工具。

Quadrant 2 contains a lot of different types of tests and activities. We need the right tools to facilitate gathering, discussing, and communicating examples and tests. Simple tools such as paper or a whiteboard work well for gathering examples if the team is co-located. More sophisticated tools help teams write business-facing tests that guide development in an executable, automatable format. In the next chapter, we’ll look at the kinds of tools needed to elicit examples, and to write, communicate, and execute business-facing tests that support the team.

概括

Summary

在本章中,我们研究了在编码过程中通过面向业务的测试来支持团队的方法。

In this chapter, we looked at ways to support the team during the coding process with business-facing tests.

图像在敏捷开发中,示例和面向业务的测试,而不是传统的需求文档,告诉团队要编写什么代码。

In agile development, examples and business-facing tests, rather than traditional requirements documents, tell the team what code to write.

图像在短期迭代中处理功能的薄片,使客户有机会查看和使用应用程序并根据需要调整他们的需求。

Working on thin slices of functionality, in short iterations, gives customers the opportunity to see and use the application and adjust their requirements as needed.

图像测试人员贡献的一个重要领域是帮助客户表达满意条件,并为每个故事创建期望和不期望的行为示例。

An important area where testers contribute is helping customers express satisfaction conditions and create examples of desired, and undesired, behavior for each story.

图像提出开放式问题以帮助客户思考所有需要的功能并防止隐藏重要假设。

Ask open-ended questions to help the customer think of all of the desired functionality and to prevent hiding important assumptions.

图像帮助客户就故事的期望行为达成共识,以适应业务不同部门的各种观点。

Help the customers achieve consensus on desired behavior for stories that accommodate the various viewpoints of different parts of the business.

图像帮助客户开发工具(例如,故事清单)来表达业务满意度等信息。

Help customers develop tools (e.g., a story checklist) to express information such as business satisfaction conditions.

图像开发和客户团队应该考虑给定故事影响的应用程序的所有部分,同时牢记整个系统的功能。

The development and customer teams should think through all of the parts of the application that a given story affects, keeping the overall system functionality in mind.

图像 与您的团队合作,将功能集分解为小的、可管理的故事和故事中的路径。

Work with your team to break feature sets into small, manageable stories and paths within stories.

图像循序渐进地遵循“编写测试—编写代码—运行测试—学习”的模式,在每次通过功能的基础上进行构建。

Follow a pattern of “write test—write code—run tests—learn” in a step-by-step manner, building on each pass through the functionality.

图像使用测试和示例来降低缺少功能或忽视全局的风险。

Use tests and examples to mitigate risks of missing functionality or losing sight of the big picture.

图像通过面向业务的测试驱动编码使开发团队不断意识到实现可测试应用程序的必要性。

Driving coding with business-facing tests makes the development team constantly aware of the need to implement a testable application.

图像支持团队的面向业务的测试必须自动化,以便快速轻松地提供反馈,以便团队可以在短迭代中交付价值。

Business-facing tests that support the team must be automated for quick and easy feedback so that teams can deliver value in short iterations.

第 9 章支持团队的面向业务的测试工具包

Chapter 9 Toolkit for Business-Facing Tests that Support the Team

图像

在上一章中,我们讨论了如何进行业务或功能测试以支持团队努力构建正确的软件。在本章中,我们将研究一些可以用来帮助您的团队在象限 2 测试中取得成功的工具。

In the previous chapter, we talked about how to approach business or functional testing to support the team in its effort to build the right software. In this chapter, we’ll examine some of the tools you can use to help your team succeed with Quadrant 2 tests.

面向业务的测试工具策略

Business-Facing Test Tool Strategy

我们如何捕获面向业务的测试来帮助程序员知道要编写什么代码?程序员和客户之间面对面的对话通常是最好的方式,但即使客户是您团队的一员,他们也没有整天与程序员闲逛并解释功能。如果任何客户或开发人员团队成员在不同的位置,即兴的走廊对话可能不可行。此外,六个月从现在开始,我们可能想要一种方法来记住我们为什么以某种方式对功能进行编码。如果我们的一些团队成员在不同的地点,我们肯定需要某种方式以电子方式共享信息。

How do we capture the business-facing tests that help the programmers know what to code? Face-to-face conversations between programmers and customers are usually the best way, but even when customers are part of your team, they don’t have all day to hang out with programmers and explain features. If any customer or developer team members are in different locations, impromptu hallway conversations might not be feasible. Besides, six months from now, we might want a way to remember why we coded a piece of functionality a certain way. If some of our team members are in different locations, we’re definitely going to need some way to share information electronically.

随着敏捷开发的流行,我们有越来越多的工具来帮助我们捕获示例并使用它们编写可执行的测试。可用的工具变化太快,我们无法在本书中列出它们的清单,但我们可以提供一些工具示例和一些使用它们的策略,以帮助提供支持团队开发新故事的面向业务的测试。我们在这里讨论的一些工具不是新的,也不是专门针对敏捷开发的,但它们在敏捷项目中运行良好。

As agile development has gained in popularity, we have more and more tools to help us capture examples and use them to write executable tests. The tools available are changing too fast for us to include an inventory of them in this book, but we can offer some examples of tools and some strategies for using them to help provide business-facing tests that support the team’s development of new stories. Some of the tools we discuss here aren’t new, or specific to agile development, but they work well in an agile project.

选择所需工具的策略应基于团队的技能组合、应用程序使用的技术、团队的自动化优先级、时间和预算限制以及您的情况所特有的其他问题。您对一个或多个工具的选择不应基于推销员提供的最新最酷的工具。您可能需要许多不同的工具来解决不同的问题。

Your strategy for selecting the tools you need should be based on your team’s skill set, the technology your application uses, your team’s automation priorities, time and budget constraints, and other concerns unique to your situation. Your selection of a tool or tools should not be based on the latest and coolest tool offered by a salesman. You might need many different tools to solve different problems.

有关测试自动化的一般方法的更多信息,请参阅第 14 章,“敏捷测试自动化策略”。

For more information about a general approach to test automation, see Chapter 14, “An Agile Test Automation Strategy.”

我们鼓励客户做一些提前准备,并准备好在迭代计划期间为每个故事解释示例。测试人员可以很好地帮助客户弄清楚如何在迭代开始时提供正确数量的细节。很难取得恰到好处的平衡。

We encourage customers to do some advance preparation and to be ready to explain examples for each story during iteration planning. Testers are in a good position to help customers figure out how to provide the right amount of detail at the beginning of the iteration. It’s hard to strike just the right balance.

丽莎的故事

Lisa’s Story

在我们的团队选择使用 FitNesse 来指定和自动化面向业务的测试后不久,我们的产品负责人和我就尝试充分利用这个新工具。我们有一个极其复杂的史诗即将到来。在史诗的第一个故事开始的迭代之前,我们花了很多时间为高度复杂的业务规则编写详细的测试用例。我们对开始开发新功能感觉很好。

Soon after our team chose to use FitNesse for specifying and automating business-facing tests, our product owner and I tried to make good use of the new tool. We had an extremely complex epic coming up. We spent many hours writing detailed test cases for highly complex business rules weeks in advance of the iteration where the first story of the epic was started. We felt good about getting a running start on developing the new functionality.

当他们开始处理这些故事时,程序员抱怨说他们无法从这些详细的测试中了解全局。测试的设计方式也与实际代码设计不兼容。我最终花了几个小时重构它们。这并不是完全浪费时间,因为至少我很好地理解了这些故事,并且我们有许多最终可以使用的测试用例,但这对我们的团队来说不是正确的方法。反复试验向我们表明,高级测试与一些期望和不期望行为的示例相结合是程序员了解从什么开始编码的最佳方式。

When they started working on these stories, the programmers complained that they couldn’t get the big picture from these detailed tests. The tests were also designed in a way that was incompatible with the actual code design. I ended up spending hours refactoring them. It wasn’t a complete waste of time, because at least I understood the stories well and we had a number of test cases we could use eventually, but it wasn’t the right approach for our team. Trial and error has shown us that high-level tests combined with a few examples of desired and undesired behavior are the best way for the programmers to know what to start coding.

——丽莎

—Lisa

在测试用例中试验不同级别的前期细节,找出最适合您的团队的方法。无论您追求何种程度的详细信息,您都需要一些方法来帮助客户找到并表达所需系统行为的示例。在下一节中,我们将了解可以执行此操作的工具类型。

Experiment with different levels of up-front detail in test cases to figure out what works best for your team. Whatever level of detail you’re after, you need some way to help customers find and express examples of desired system behavior. In the next section, we look at the types of tools that can do that.

引出示例和需求的工具

Tools to Elicit Examples and Requirements

正如我们在第 8 章中指出的那样,故事只是就所需行为进行长时间对话的起点。拥有正确大小的故事,其中明确说明了功能、用户和目的,这让我们有一个良好的开端。它们不是很详细,因为正如 Mike Cohn [ 2004 ] 指出的那样,最好推迟收集细节,直到故事包含在迭代中。为一个可能永远不会被收录的故事收集细节是一种资源浪费。我们喜欢 Mike Cohn 在User Stories Applied中描述的用户故事的“角色、功能、业务价值”模式,如下所示:

As we pointed out in Chapter 8, stories are only a starting place for a prolonged conversation about the desired behavior. Having correctly sized stories where the feature, user, and purpose are clearly stated gives us a head start. They aren’t very detailed, because as Mike Cohn [2004] points out, it’s best to defer collecting details until the story is included in an iteration. Collecting details for a story that might never be included is a waste of resources. We like the “role, function, business value” pattern for user stories that Mike Cohn describes in User Stories Applied, as in:

作为一个(角色),我想要(功能)以便(商业价值)。

As a (role), I want (function) so that (business value).

这种格式并不适合所有人,因此我们鼓励您进行试验,看看哪种格式最适合您的情况。无论您的用户故事如何阅读,您都需要某种方式通过示例和面向业务的测试来指导开发来充实这些故事。

This format doesn’t work for everyone, so we encourage you to experiment and see what works best in your situation. Regardless of how your user stories read, you need some way to flesh those stories out with examples and business-facing tests that guide development.

一个简单的故事可以产生广泛的影响,不仅对应用程序,而且对整个组织、它的客户、它的同事、供应商或合作伙伴。如果我们更改 API,我们必须通知可能正在使用它的任何客户或供应商。如果我们计划更改 UI,我们希望,甚至可能有合同义务,提前向用户发出一定数量的通知。故事可能会影响法律问题或影响外部报告。新功能通常意味着新的或更新的文档。当然,更改的功能可能会影响系统的其他部分。

One simple story can have a wide-ranging impact, not only on the application, but across the organization, its clients, its associates, vendors, or partners. If we change an API, we have to notify any customers or vendors who might be using it. If we plan a UI change, we want, or might even be contractually obligated, to give a certain amount of advance notice to users. Stories may affect legal concerns or impact external reporting. New features often mean new or updated documentation. Of course, changed functionality is likely to affect other parts of the system.

包括测试人员在内的软件开发团队应该帮助客户捕获和传达与每个故事或主题相关的所有需求。开发新功能,只是因为法律原因或因为业务合作伙伴没有及时通知而被阻止发布它们,这是一种令人沮丧的时间浪费(问问 Lisa!)。精益开发教会我们在开发软件时避免浪费。

The software development team, including the testers, should help the customer capture and communicate all of the requirements related to each story or theme. Developing new features, only to be prevented from releasing them for legal reasons or because a business partner wasn’t informed in time, is a frustrating waste of time (just ask Lisa!). Lean development teaches us to avoid waste while we develop software.

哪些工具可以帮助我们通过示例说明所需的行为、集思广益潜在的实施和连锁反应,以及创建可以转化为测试的需求?一些例子是:

What tools can help us illustrate desired behavior with examples, brainstorm potential implementations and ripple effects, and create requirements we can turn into tests? Some examples are:

该列表包括许多简单的工具,它们不是敏捷测试所独有的,但不应被忽视。在敏捷开发中,简单的解决方案通常是最好的。让我们更详细地看看这些。

The list includes a number of simple tools that aren’t unique to agile testing but that shouldn’t be neglected. In agile development, simple solutions are usually best. Let’s look at these in more detail.

清单

Checklists

清单是产品所有者确保他们正确评估和传达故事所有方面的一种方式。Lisa 团队的产品负责人 Steve Perkins 提出了他自己的“故事清单”,以确保他和利益相关者仔细考虑受故事影响的一切。为此,他在团队 wiki 上创建了一个模板。清单指定了满意的条件——业务需要从故事中得到什么。它还包括对现有功能的影响,例如网站、文件、管理表格、账户报表和系统的其他组件以及业务的日常运营。清单确保团队不会因为忘记考虑而错过诸如数据迁移、通知、法律考虑以及与供应商和业务合作伙伴的沟通等要求。图 9-1显示了一个示例故事清单。

Checklists are one way for product owners to make sure they correctly assess and communicate all of the aspects of a story. The product owner for Lisa’s team, Steve Perkins, came up with his own “story checklist” to make sure he and the stakeholders think through everything affected by the story. He created a template on the team wiki for this purpose. The checklist specifies the conditions of satisfaction—what the business needs from the story. It also includes impacts on existing functions such as the website, documents, administrative forms, account statements, and other components of the system and the daily operation of the business. The checklist makes sure the team doesn’t miss requirements such as data migration, notifications, legal considerations, and communications to vendors and business partners because they forgot to consider them. Figure 9-1 shows a sample story checklist.

图 9-1样本故事清单

Figure 9-1 Sample story checklist

图像

思维导图

Mind Maps

思维导图是一种简单但有效的方法,可以用来找出您在简单的头脑风暴会议中可能不会想到的想法。思维导图是为表示与中心关键概念相关联的概念、词语或想法而创建的图表。我们使用思维导图来组织这本书。

Mind maps are a simple but effective way to search out ideas that that might not occur to you in a simple brainstorming session. Mind maps are diagrams created to represent concepts, words, or ideas linked to a central key concept. We used mind maps to organize this book.

无论您是购买我们使用的工具还是在白板或大纸上绘图,这都无关紧要。效果是一样的。思维导图使您能够以与您思考问题的方式一致的方式产生想法和工作。

It really doesn’t matter whether you purchase a tool such as the one we used or draw on a whiteboard or a big piece of paper. The effect is the same. Mind maps enable you to generate ideas and work in a way that is consistent with the way you think about problems.

举个例子怎么样?我们正在讨论图 9-2中所示的故事。

How about an example? We’re discussing the story shown in Figure 9-2.

图 9-2购物车删除故事

Figure 9-2 Shopping cart delete story

图像

我们聚集在白板周围,开始提问。删除的项目应该去哪里?它们应该被保存以供以后批准,还是应该消失?删除项目后屏幕应该是什么样的?图 9-3显示了我们可能在白板上绘制的思维导图类型的示例。

We gather around the whiteboard and start asking questions. Where should the deleted items go? Should they be saved for later approval, or should they just disappear? What should the screen look like after we delete an item? Figure 9-3 shows an example of the sort of mind map we might draw on a whiteboard.

图 9-3购物车删除故事的示例思维导图

Figure 9-3 Example mind map for shopping cart delete story

图像

电子表格

Spreadsheets

如果可能,用于指定面向业务的测试的工具应该适合您的业务领域。例如,电子表格被金融服务公司广泛使用,因此对于金融服务领域的项目,使用电子表格来定义故事应该提供的功能示例是有意义的。

When possible, tools for specifying business-facing tests should fit well with your business domain. For example, spreadsheets are widely used by financial services companies, so for a project in the financial services area it makes sense to use spreadsheets to define examples of the functionality that a story should deliver.

客户可以编写一些高级测试用例来帮助在迭代开始之前完善故事,可能使用某种类型的检查表。一些客户团队只是在每张故事卡的背面写几个测试,可能是一个快乐的路径和一个负面的测试。有些人会在电子表格或他们喜欢使用的任何格式中编写更详细的示例。

Customers can write a few high-level test cases to help round out a story prior to the start of the iteration, possibly using some type of checklist. Some customer teams simply write a couple of tests, maybe a happy path and a negative test, on the back of each story card. Some write more detailed examples in spreadsheets or whatever format they’re comfortable working with.

Lisa 团队的产品负责人 Steve Perkins 经常在电子表格中说明复杂的计算和算法,团队可以稍后将其转化为测试。图 9-4显示了他的一个工作表,该工作表对输入值执行计算以生成 ADR 和 ACR 列中的值。这种格式很容易进入自动化测试框架(相应的 FitNesse 示例参见图 9-8 )。

Steve Perkins, the product owner for Lisa’s team, often illustrates complex calculations and algorithms in spreadsheets, which the team can turn into tests later. Figure 9-4 shows one of his worksheets, which performs calculations on the input values to produce the values in the ADR and ACR columns. This format is easy to get into an automated test framework (refer to Figure 9-8 for the corresponding FitNesse example).

图 9-4来自产品负责人的电子表格示例

Figure 9-4 Spreadsheet example from product owner

图像

查看您的业务专家已经使用的工具,看看它们是否可以调整以记录所需功能行为的示例,以帮助开发团队更好地理解故事。

Look at tools already used by your business experts and see whether they can be adapted to document examples of desired feature behavior to help the development team better understand the story.

Janet 曾与多个使用电子表格作为 Fit 测试输入的团队合作。这允许客户使用他们熟悉的工具工作,但不会浪费任何精力将它们转换为自动化工具。

Janet has worked with several teams that have used spreadsheets as input into their Fit tests. This allows customers to work in a tool that is familiar to them but not waste any effort in translating them to an automation tool.

模型

Mock-Ups

模型可以采用多种形式。纸质原型是测试屏幕如何协同工作的一种简单但有效的方法。在白板上画图可以达到同样的目的,但不能传来传去。现有应用程序的屏幕截图可以构成讨论如何添加新功能以及新功能适合 UI 的位置的基础。您可能在其他开发方法中使用过此类工具。敏捷开发的最大区别在于我们在开始编写代码时创建和讨论模型,而不是提前数周或数月。我们可以确信模型代表了客户现在的需求。

Mock-ups can take many forms. Paper prototypes are a simple but effective way to test how screens will work together. Drawing on a whiteboard can accomplish the same goal, but it can’t be passed around. Screenshots from existing applications can form the basis of a discussion about how to add a new feature and where it will fit into the UI. You may have used tools like these in other development methodologies. The big difference in agile development is that we create and discuss the mock-ups just as we’re about to start writing the code, rather than weeks or months beforehand. We can be confident that the mock-up represents what the customers want right now.

请参阅第 8 章,了解 Gerard Meszaros 对使用纸质原型和绿野仙踪测试的描述。

See Chapter 8 for Gerard Meszaros’ description of using paper prototypes and Wizard of Oz testing.

丽莎的故事

Lisa’s Story

我们使用简单的方法来创建模型,这样我们就不会在完成模型之前投入时间编码。通常,我们在白板上绘制 UI 或工作流,然后将其拍照上传到我们的团队 wiki,以便我们的远程团队成员也可以看到它。在其他时候,客户或我们的产品所有者在纸上绘制模型或修改现有的 UI 页面或报告以显示应添加和更改的内容。纸质模型被扫描并张贴在维基上。

We use simple approaches to creating mock-ups so that we aren’t tempted to invest time coding before we’re finished working through the mock-up. Often, we draw a UI or workflow on the whiteboard and then take photos of it to upload to our team wiki so our remote team member can also see it. At other times, a customer or our product owner draws the mock-up on paper or modifies an existing UI page or report to show what should be added and changed. The paper mock-ups are scanned in and posted on the wiki.

一张图片胜过千言万语,即使在敏捷软件开发中也是如此。模型可能比叙述更清楚地显示客户的需求。它们为讨论所需的代码行为提供了一个很好的焦点。

A picture’s worth a thousand words, even in agile software development. Mock-ups show the customer’s desires more clearly than a narrative possibly could. They provide a good focal point for discussing desired code behavior.

——丽莎

—Lisa

图 9-5显示了 Lisa 的团队用来模拟新报告的模型示例——只需标记一个相似的现有报告即可。

Figure 9-5 shows an example of a mock-up that Lisa’s team used to mock up a new report—simply by marking up an existing report that’s similar.

图 9-5示例报告模型

Figure 9-5 Sample report mock-up

图像

模型不需要花哨或漂亮,也不需要花费很多时间来创建。它们确实需要让客户和开发团队都能理解。

Mock-ups don’t need to be fancy or pretty, or to take a lot of time to create. They do need to be understandable to both the customer and developer teams.

流程图

Flow Diagrams

简单的图表工具很有用,无论团队是否在同一地点。以更永久的形式捕获讨论期间制定的工作流或决策树通常是个好主意。流程图可以成为用户场景的基础,可以帮助您将两个或三个用户故事联系在一起。让我们再看看我们在第 8 章中介绍的装运订单故事(见图9-6)。

Simple diagramming tools are helpful, whether the team is co-located or not. It’s often a good idea to capture in a more permanent form a workflow or decision tree worked out during a discussion. Flow diagrams can become the basis of a user scenario that might help you tie two or three user stories together. Let’s look at the shipping order story again that we introduced in Chapter 8 (see Figure 9-6).

图 9-6运费故事

Figure 9-6 Story for shipping charges

图像

图 9-7显示了一个非常简单的决策过程流程图,该过程根据阈值订单金额确定客户的订单是否符合免费送货条件。因为我们已经与我们的客户讨论过这个故事,所以我们发现客户的订单不仅必须超过美元金额阈值,而且必须只发送到一个地址,而且它的重量必须小于运输重量阈值。如果满足所有这些条件,客户的订单将免费发货;否则,客户将不得不从“选择运输选项”页面进行选择。

Figure 9-7 shows a very simple flowchart of a decision process for whether a customer’s order is eligible for free shipping based on a threshold order amount. Because we’ve discussed this story with our customer, we’ve found out that the customer’s order must not only exceed a threshold dollar amount but also must be to one address only, and it must weigh less than a shipping weight threshold. If all of these conditions are satisfied, the customer’s order will ship free; otherwise, the customer will have to select from the “choose shipping options” page.

图 9-7获得免费送货选项资格的流程图

Figure 9-7 Flow chart for qualifying for free shipping option

图像

流程图和思维导图等视觉效果是描述故事功能概览的好方法,尤其是当它们由一组客户、程序员和测试人员绘制时。在敏捷开发中,我们在开始编写测试和代码时创建这些图表。根据这些,团队可以立即开始深入挖掘详细需求。

Visuals such as flow diagrams and mind maps are good ways to describe an overview of a story’s functionality, especially if they’re drawn by a group of customers, programmers, and testers. In agile development, we create these diagrams as we’re about to start writing tests and code. From these, the team can immediately start digging down to the detailed requirements.

基于软件的工具

Software-Based Tools

如果我们与客户处于不同的位置,我们需要工具来帮助我们与他们交谈。分布式团队告诉我们,桌面共享是帮助他们处理在不同地点工作的首要工具。Windows NetMeeting 和 VNC 是让两个团队成员在不同位置进行配对测试的工具示例。WebEx 和 Skype 等视频会议工具支持远程团队和客户之间的协作和演示。Scriblink 等在线白板和 Mimeo 等交互式白板工具有助于分布式白板讨论。

If we’re in a different location than our customers, we need tools to help us converse with them. Distributed teams tell us that desktop sharing is the number one tool that helps them deal with working in separate locations. Windows NetMeeting and VNC are examples of tools that let two team members in different locations pair-test. Video conferencing tools such as WebEx and Skype enable collaboration and demos between remote teams and customers. Online whiteboards such as Scriblink and interactive whiteboard tools such as Mimeo facilitate distributed whiteboard discussions.

更多适合产品所有者和业务专家直接使用的工具正在变得可用,并且许多团队开发了自己的工具。Fit(集成测试框架)和 FitNesse 等工具旨在促进客户与开发团队之间的协作和沟通。我们听说有更多团队的客户实际使用诸如此类的工具编写测试。

More tools that are geared for direct use by product owners and business experts are becoming available, and many teams develop their own. Tools such as Fit (Framework for Integrated Tests) and FitNesse were designed to facilitate collaboration and communication between the customer and development teams. We’re hearing about more teams where the customers actually write the tests in a tool such as those.

一些团队构建自己的框架,允许客户、业务分析师和测试人员记录可以直接转化为可执行测试的示例。这些通常基于开源工具,例如 xUnit、Fit、Selenium 和 Watir。我们喜欢这种方法,因为它可以节省时间和资源。当您在短迭代中交付生产就绪代码时,您需要一个简化的流程。

Some teams build their own frameworks that allow customers, business analysts, and testers to document examples that can be directly turned into executable tests. These are often based on open source tools such as xUnit, Fit, Selenium, and Watir. We like this approach, because it saves time and resources. When you’re delivering production-ready code in short iterations, you need a streamlined process.

在线论坛工具是电子邮件对话的一个很好的替代方案,可以持续讨论功能或技术问题,尤其是对于并非所有人都坐在一起的团队。电子邮件经常被遗漏或丢失,人们必须记住选择“全部回复”,而且以后很难整理讨论的细节。Lisa 的团队使用在线论坛来征求对不同工具的意见,为功能提出不同的行为,并进行哲学讨论,例如是否跟踪缺陷。

Online forum tools are a good alternative to email conversations for ongoing discussions about features or technical concerns, especially for teams that don’t all sit together. Emails often get missed or lost, people have to remember to choose “Reply all,” and it can be hard to put together the details of the discussion later. Lisa’s team uses an online forum to elicit opinions about different tools, propose different behavior for features, and conduct philosophical discussions such as whether to track defects.

找到合适的电子工具对于分布式团队尤为重要。即时消息、电话、VoIP 和 Skype 帮助我们进行交流,但它们缺乏可视化组件。一些全球团队要求他们的成员在非标准时间开会,以便他们可以进行实时对话,但书面和视觉交流的框架仍然至关重要。

Finding the right electronic tools is particularly vital for distributed teams. Instant messaging, the telephone, VoIP, and Skype help us communicate, but they lack the visual component. Some global teams ask their members to meet at nonstandard hours so that they can have real-time conversations, but frameworks for written and visual communication are still critical.

Wiki 是一种常用工具,用于加强交流和记录讨论和决策。Wiki 使用户能够在 Web 浏览器中编辑网页内容。用户可以添加超链接并轻松创建新页面。您可以上传白板绘图的模型、示例和图片,并使它们在 Wiki 页面上易于显示。分层组织可能难以维护,但是有许多可用的开源和供应商 wiki 软件包可以使管理您的知识库和共享信息更容易管理。如果您的 wiki 知识库已经发展到难以找到任何东西的地步,请聘请技术作家将其转化为有条理的可用文档。

Wikis are a common tool used to enhance communication and record discussions and decisions. Wikis enable users to edit web page content in a web browser. Users can add hyperlinks and easily create new pages. You can upload mock-ups, samples, and pictures of whiteboard drawings and make them easily visible on Wiki pages. The hierarchical organization can get tricky to maintain, but there are lots of open source and vendor wiki software packages available that make managing your knowledgebase and sharing information easier to administer. If your wiki knowledgebase has grown to the point where it’s hard to find anything, hire a technical writer to transform it into organized, usable documentation.

开源和商业工具提供了让团队在线协作处理需求和测试用例的方法。我们怎么强调都不够强调您需要确定可能有用的工具,对它们进行几次迭代试验,并确定它们对您的工作效果如何。您团队的需求会随着时间而改变,因此请始终乐于尝试新技术和框架。

Open source and commercial tools provide ways to let teams collaborate on requirements and test cases online. We can’t emphasize enough the need for you to identify tools that might be helpful, to experiment with them for a few iterations, and to decide how well they work for you. Your team’s needs will change with time, so always be open to trying new techniques and frameworks.

这些工具有助于创建关于故事的对话。借助这些技术,以及尽可能多的实时对话和视觉共享,我们可以从一开始就定义正确的产品。

These tools help create the conversation about the story. With these techniques, and as much real-time conversation and visual sharing as we can manage, we can define the right product from the get-go.

基于示例的自动化测试工具

Tools for Automating Tests Based on Examples

测试工具呢?我们喜欢 Fit 和 FitNesse 等工具固有的协作功能。然而,在我们看来,任何能吸引测试人员和程序员的工具,程序员和客户,以及测试人员和客户交谈是一件很棒的事情。我们知道客户实际使用 Fit、FitNesse、Expect 或其他工具编写测试的团队。当工具的设置方式对每个编写测试的人来说都很清楚,并且域语言易于理解并提供适当的固定装置时,这就会起作用。

What about test tools? We like the collaboration inherent with tools such as Fit and FitNesse. However, in our opinion, any tool that gets testers and programmers, programmers and customers, and testers and customers talking is a great one. We know teams where customers actually write tests in Fit, FitNesse, Expect, or other tools. This works when the tool has been set up in a manner that’s clear to everyone writing tests, with the domain language easy to understand and the appropriate fixtures provided.

在 GUI 和 API 级别下测试的工具

Tools to Test below the GUI and API Level

有许多开源工具可让您在 GUI 下或 API 层进行测试。我们仅列出了几个,但您的团队需要为您确定合适的工具。

There are a multitude of open source tools that enable you to test below the GUI or at the API layer. We are listing just a few, but your team will need to determine the right tool for you.

单元级测试工具

一些团队使用 xUnit 工具(如 JUnit 或 NUnit)进行面向业务的测试和面向技术的测试。如果测试人员和客户对这些工具感到满意,并且它们提供了所需的 GUI 背后的所有功能测试,那么它们就很好。为了使这些工具对客户更加友好,团队可能会在单元级工具之上构建一个框架,测试人员和客户可以使用该框架来指定测试。

Some teams use the xUnit tools such as JUnit or NUnit for business-facing tests as well as technology-facing tests. If the testers and customers are comfortable with these tools, and they provide for all of the functional testing behind the GUI needed, they’re fine. To make these tools more customer-friendly, teams might build a framework on top of the unit-level tools that testers and customers can use to specify tests.

Janet 开发过几个这样的应用程序。一个是在组织中部署的消息处理系统。程序员使用 JUnit 进行所有组件和集成测试。他们构建了一个可以使用 JUnit 测试的负载测试框架,因此不需要其他测试工具。GUI 前端非常小,Janet 能够手动测试它。在这种情况下,自动化 GUI 测试毫无意义。

Janet has worked on a couple of applications like that. One was a message handling system that was being deployed in an organization. The programmers used JUnit for all of the component and integration testing. They built a load test framework that could make use of the JUnit tests, so no other testing tools were needed. The GUI front end was so small that Janet was able to test it manually. It made no sense to automate the GUI testing in this case.

行为驱动开发 (BDD) 工具也适用于此目的,因为它们使用更自然的语言来指定测试。行为驱动开发是测试驱动开发的变体,由 Dan North [ 2006 ] 首创,并由许多其他人发展而来。它与领域驱动设计有关,关注领域而不是技术,并用模型驱动设计。BDD 没有使用“测试”或“断言”一词,而是使用了“应该”一词。通过从行为的角度考虑,在代码之前编写规范是很自然的。测试规范使用特定领域的语言来提供客户可以阅读但也可以轻松实现自动化的测试。

Behavior-driven development (BDD) tools are also suited to this purpose, because they use a more natural language for specifying the tests. Behavior-driven development is a variation of test-driven development, pioneered by Dan North [2006], and evolved by many others. It’s related to domain-driven design, with a focus on the domain rather than on the technology, and driving design with a model. Instead of the word “test” or “assert,” BDD uses the word “should.” By thinking in terms of behavior, it’s natural to write specifications ahead of code. Test specifications use a domain-specific language to provide tests that customers can read but that can also be easily automated.

在撰写本文时可用的许多 BDD 工具中的一些包括用于 Java 平台的 easyb 和 JBehave,用于 .NET 的 NBehave 和 NSpec,以及用于 Ruby 的 RSpec。这些工具,如 XUnit 工具,旨在供程序员用来指导编码,但它们也可用于表达推动开发的面向业务的测试,让客户更密切地参与开发过程。

Some of the many BDD tools available as of this writing include easyb and JBehave for the Java platform, NBehave and NSpec for .NET, and RSpec for Ruby. These tools, like the XUnit tools, are intended for use by programmers to guide coding, but they can also be used to express business-facing tests that drive development, involving customers more closely in the development process.

支持团队的面向业务的测试的目标是促进客户和开发人员之间的沟通和协作,并使团队能够在每次迭代中交付真正的价值。一些团队使用单元级工具在这方面做得最好,而其他团队则更好地适应功能级测试工具。

The goal of business-facing tests that support the team is to promote communication and collaboration between customers and developers, and to enable teams to deliver real value in each iteration. Some teams do this best with unit-level tools, and others adapt better to functional-level test tools.

API层功能测试工具

在 Lisa 加入她的第一个敏捷团队之前,“在 GUI 后面”进行测试是一个听起来不错的概念,但她从未有机会尝试过。Fit 和建立在 Fit 之上的 FitNesse 是功能测试工具,源于客户团队需要能够编写和理解推动开发的面向业务的测试。使用这些工具,团队可以在不涉及表示层的情况下测试业务逻辑。

Before Lisa joined her first agile team, testing “behind the GUI” was a concept that sounded good, but she’d never had the opportunity to try it. Fit, and FitNesse, which is built on top of Fit, are functional test tools that grew from the need for the customer team to be able to write and understand the business-facing tests that drive development. With these tools, teams can test business logic without involving the presentation layer.

Fit 和 FitNesse。Fit (Framework for Integrated Tests) 是一个促进协作的开源测试框架,这使其成为帮助细化需求的好工具。Fit 是 Ward Cunningham 的发明,享有杰出的开发人员名单。Fit 使客户、测试人员和程序员能够使用示例来指定他们期望系统执行的操作。测试运行时,Fit 会自动将客户的期望值与实际结果进行比较。

Fit and FitNesse. Fit (Framework for Integrated Tests) is an open source testing framework that promotes collaboration, which makes it a good tool to help refine requirements. The invention of Ward Cunningham, Fit has enjoyed an illustrious roster of contributing developers. Fit enables customers, testers, and programmers to use examples to specify what they expect the system to do. When the tests run, Fit automatically compares customers’ expectations to actual results.

借助 Fit,客户可以使用他们的主题专业知识提供指导,以定义程序员可以编码的示例。程序员通过编写对示例进行实际检查的固定装置来参与。这些装置使用示例中指定的数据与实际程序一起运行。

With Fit, customers can provide guidance using their subject matter expertise to define the examples that the programmers can code against. The programmers participate by writing the fixtures that do the actual checks against the examples. These fixtures use the data specified in the examples to run with the actual program.

适合性测试由将测试输入传递给生产代码然后接受输出并与预期结果进行比较的夹具自动执行。测试结果用颜色编码,因此很容易发现故障或异常。

Fit tests are automated by fixtures that pass the test inputs to the production code and then accept the outputs, which it then compares with expected results. The test results are color-coded, so it’s easy to spot a failure or exception.

Fit 测试以 HTML 表格的形式编写,但团队可以自定义 Fit,以便可以将测试编写为电子表格或客户、测试人员和分析师认为可用的任何形式。

Fit tests are written as HTML tables, but teams can customize Fit so that tests can be written in spreadsheets or whatever form the customers, testers, and analysts find usable.

在fit.c2.com上了解有关 Fit 的更多信息。

Learn more about Fit at fit.c2.com.

FitNesse 是一个基于 Fit 的 Web 服务器、wiki 和软件测试工具。最初由 Robert C. “Uncle Bob” Martin 和 Micah Martin 开发,它是一个拥有活跃开发者社区的开源工具。FitNesse 和 Fit 之间的主要区别是 FitNesse 测试是用 wiki 标记而不是 HTML 表格编写的,一些用户认为后者更容易。它还支持在电子表格中创建测试并将其导入到测试中。

FitNesse is a web server, a wiki, and a software testing tool that is based on Fit. Originally developed by Robert C. “Uncle Bob” Martin and Micah Martin, it’s an open source tool with an active developer community. The main difference between FitNesse and Fit is that FitNesse tests are written in wiki markup instead of HTML tables, which some users find easier. It also supports creating tests in spreadsheets and importing those into the tests.

在www.fitnesse.org上了解有关 FitNesse 的更多信息。

Learn more about FitNesse at www.fitnesse.org.

图 9-8显示了从图 9-4中的示例构建的 FitNesse 测试的一部分。添加了更多输入以使生产代码运行,但基本测试数据来自电子表格。测试结果通过时颜色编码为绿色,失败时为红色。

Figure 9-8 shows part of the FitNesse test that was built from the example in Figure 9-4. More inputs were added to make the production code run, but the essential test data is from the spreadsheet. The test results are color-coded green when they pass, red when they fail.

图 9-8来自客户示例的自动 FitNesse 测试

Figure 9-8 Automated FitNesse test from customer example

图像

Fit 或 FitNesse 类型的工具的另一个好处是它促进不同团队成员之间的协作,以便提出正确的测试来指导发展。客户、程序员、测试人员和其他人一起工作来指定和自动化测试。

Another benefit of a Fit or FitNesse type of tool is that it promotes collaboration among different team members in order to come up with the right tests to guide development. Customers, programmers, testers, and others work together to specify and automate the tests.

测试网络服务。Web 服务只是 API 的另一种形式,它使其他应用程序能够访问您的应用程序。让我们谈谈可用于测试系统各种输入的一些工具。

Testing Web Services. Web services is just another form of an API that enables other applications to access your application. Let’s talk about some of the tools you can use to test various inputs into your system.

交叉检查。CrossCheck 是一种用于测试 Web 服务的工具示例。您提供WSDL(Web 服务描述语言);CrossCheck 编译该页面,然后为您呈现一个选项卡式菜单,其中包含供您填写的文本框。它有一个运行模式,您可以在其中将测试添加到套件中,然后运行该套件。Lisa 和 Janet 都没有尝试过这个工具,但是在 Yahoo 敏捷测试组中,它被认为是一个用于测试 Web 服务的工具,如果你每次都运行相同的数据的话。

CrossCheck. CrossCheck is one example of a tool for testing web services. You supply the WSDL (Web Services Description Language); CrossCheck compiles the page and then presents you with a tabbed menu that contains textboxes for you to fill in. It has a Run mode where you can add your tests to a suite and then run the suite. Neither Lisa or Janet have tried this tool, but it was noted on the Yahoo agile-testing group as a tool to use for testing web services if you were running the same data through each time.

Ruby 测试::单元。Janet 参与的一个项目使用 Ruby 的单元测试框架 Test::Unit 来测试 Web 服务,并取得了巨大成功。事实上,该团队能够尽早进行测试,为程序员提供即时反馈,这有助于最终设计。

Ruby Test::Unit. One project Janet was on used Ruby’s unit testing framework, Test::Unit, to test web services, with great success. In fact, the team was able to test early to give the programmers immediate feedback, which helped with the final design.

请参阅第 12 章“测试象限总结”中的“系统测试”示例,了解 Janet 的团队如何使用 Ruby Test::Unit 来测试 Web 服务。

See the “System Test” example in Chapter 12, “Summary of Testing Quadrants,” to see how Janet’s team used Ruby Test::Unit to test web services.

肥皂用户界面。建议用于测试 Web 服务的另一个工具是 soapUI。它具有陡峭的学习曲线,但可用于性能和负载测试。因为它可以遍历 Excel 电子表格或文本文件中的行,所以它可以用于数据驱动的测试。

soapUI.Another tool suggested for testing web services is soapUI. It has a steep learning curve but can be used for performance and load testing. Because it can loop though rows in an Excel spreadsheet or text file, it can be used for data-driven testing.

在表示层以下的层工作的测试非常适合编写和自动化指导编码的客户测试。一些从业者还没有从故事测试驱动开发中获得他们期望的价值。Brian Marick [ 2008 ] 假设使用程序员测试驱动开发构建的应用程序、大量示例的面向业务的设计严重依赖于白板讨论、一小组自动完整性测试和大量探索性测试可能更便宜且同样有效的方法。无论采用哪种方法,如果您要测试带有用户界面的应用程序,您都需要在 GUI 级别实现一些自动化。

Tests that work at the layers below the presentation layer are well suited for writing and automating customer tests that guide coding. Some practitioners haven’t gotten the value they expected from story test-driven development. Brian Marick [2008] hypothesized that an application built with programmer test-driven development, example-heavy business-facing design that relies heavily on whiteboard discussions, a small set of automated sanity tests, and lots of exploratory testing could be a less expensive and equally effective approach. Whichever approach you take, if you’re testing an application with a user interface, you’ll need some automation at the GUI level.

通过 GUI 进行测试的工具

Tools for Testing through the GUI

等一下。我们如何使用 GUI 测试来驱动开发,因为 GUI 在故事完成之前不会准备好?这听起来有悖常理,但自动化 GUI 测试对我们开发新功能非常重要。测试框架可用于在编写代码之前为 GUI 工具指定测试用例。此外,您可以在之前自动化 GUI 测试通过使用 HTML 模型或通过所有屏幕开发一个端到端的准系统切片来完成编码,这些屏幕只是简单地导航但尚未提供所有功能。即使您没有使用大量自动化故事测试来推动开发,但如果没有自动化的任何帮助,帮助我们了解功能并提供即时反馈的手动探索性测试也会变得非常乏味和缓慢。让我们看看有助于使用面向业务的测试推动开发的 GUI 测试工具的类型。

Wait a minute. How can we use GUI tests to drive development, because the GUI won’t be ready until the story is complete? It sound counterintuitive, but automated GUI tests are important to help us while we’re developing new functionality. Test frameworks can be used to specify test cases for a GUI tool before the code is written. In addition, you can automate GUI tests before coding is finished, either by using HTML mock-ups or by developing an end-to-end bare-bones slice through all of the screens that simply navigates but doesn’t provide all of the functionality yet. Even if you’re not using a lot of automated story tests to drive development, manual exploratory testing that helps us learn about the functionality and provides immediate feedback gets pretty tedious and slow without any assistance from automation. Let’s look at the types of GUI test tools that help drive development using business-facing tests.

录制/回放工具

录制/回放工具很有吸引力,因为您通常可以学会如何录制脚本并快速回放,并且可以在短时间内创建大量脚本。然而,它们也有缺点。早期的 GUI 测试工具使用 XY 屏幕坐标记录鼠标移动。使用这些工具的脚本也可能对屏幕分辨率、颜色深度甚至窗口在屏幕上的放置位置的变化很敏感。

Record/playback tools are appealing because you can usually learn how to record a script and play it back quickly, and you can create lots of scripts in a short time. However, they have drawbacks. Early GUI test tools recorded mouse movements using X-Y screen coordinates. Scripts using those tools might also be sensitive to changes in screen resolution, color depth, and even where the window is placed on the screen.

大多数现代 GUI 测试工具都使用对象来识别图形应用程序中的控件,例如按钮、菜单和文本输入小部件,因此它们可以通过符号而不是原始屏幕坐标来引用它们。这使得应用程序更易于测试,因为它更强大地经受住变化。一个按钮可能会移动到屏幕的不同部分,但测试仍然可以根据它的对象名称找到它。

Most modern GUI test tools use objects to recognize the controls in a graphical application, like buttons, menus, and text input widgets, so they can refer to them symbolically rather than with raw screen coordinates. This makes the application much more testable, because it’s more robust standing up to changes. A button might move to a different part of the screen, but the test can still find it based on its object name.

即使改进了对象识别,使用记录/回放创建的脚本通常也很脆弱且维护成本高。录制可能是开始创建脚本的好方法。了解工具脚本语言的测试人员或程序员可以将录制的脚本重构为更易于使用和维护的面向对象模型。从历史上看,记录/回放工具使用专有的脚本语言,程序员对学习这些语言不感兴趣。更改测试中使用的设计模式也更加困难。

Even with improved object recognition, scripts created with record/playback are usually brittle and expensive to maintain. Recording can be a good way to start creating a script. Testers or programmers who know the tool’s scripting language can refactor the recorded script into an object-oriented model that’s easier to use and maintain. Historically, record/playback tools used proprietary scripting languages, which programmers aren’t interested in learning. It’s also more difficult to change the design patterns used in the tests.

一些基于脚本的工具(例如我们将在接下来几节中讨论的工具)提供记录功能,以帮助人们快速开始编写测试脚本。但是,使用这些工具,录制的脚本并不是为了直接播放而设计的;它们只是创建设计良好且易于维护的测试套件的起点。

Some script-based tools such as the ones we’ll talk about in the next few sections offer a record feature to help people get a quick start on writing the test script. However, with those tools, the recorded scripts aren’t intended for straight playback; they’re just a starting point to creating a well-designed and easily maintained suite of tests.

许多敏捷团队更喜欢可以让他们创建自己的领域特定语言 (DSL) 的工具和脚本语言。这使得业务专家更容易理解甚至编写测试。接下来让我们看看其中的一些。

Many agile teams prefer tools and scripting languages that let them create their own domain-specific language (DSL). This makes tests much easier for business experts to understand and even write. Let’s look at some of these next.

敏捷开源测试工具

本节中的每个工具最初都是由一个敏捷开发团队编写的,该团队需要一个 GUI 测试工具并且找不到任何适合其情况的第三方工具。使用这些工具,您可以编写像人类用户一样使用 Web 应用程序的脚本。他们填写文本字段,从列表中进行选择,然后单击复选框和按钮。它们提供了多种方法来验证页面的正确导航和内容,例如特定于工具的验证步骤或 XPath。其中一些工具的学习曲线比简单的录制/回放工具更高,但额外的时间投资通常会在总拥有成本较低的脚本中得到回报。

Each of the tools in this section was originally written by an agile development team that needed a GUI test tool and couldn’t find any third-party tools that worked for its situation. With these tools, you can write scripts that use web applications just like a human user. They fill in text fields, select from lists, and click checkboxes and buttons. They provide a variety of ways to verify correct navigation and contents of pages, such as tool-specific verify steps or XPath. Some of these tools have a higher learning curve than simple record/playback tools, but the extra investment of time usually pays off in scripts with a low total cost of ownership.

Ruby 与 Watir。Watir(Ruby 中的 Web 应用程序测试)是一个简单的开源 Ruby 库,用于自动化与 Windows 上的 Internet Explorer 配合使用的 Web 浏览器。其他浏览器有不同的风格,包括用于 Firefox 的 FireWatir 和用于 Safari 的 SafariWatir。

Ruby with Watir. Watir (Web Application Testing in Ruby) is a simple open source Ruby library for automating web browsers that works with Internet Explorer on Windows. There are different flavors for other browsers, including FireWatir for Firefox and SafariWatir for Safari.

珍妮特的故事

Janet’s Story

我参与了一个使用 Ruby 和 Watir 开发三层测试框架的项目。第一层是一组通用的库,第二层是访问页面并提供导航。第三层和顶层使用映射到业务需求的固定类型方法创建了域语言。这允许手动测试人员在编码完成之前为工作流编写高级自动化测试。如果由于新功能而导致夹具不存在,则可以创建测试,并且可以“模拟”缺少夹具的动作词。一旦对夹具进行编码,就可以将测试作为验收测试运行。

I worked on a project that developed a three-layer test framework using Ruby and Watir. The first layer was a common set of libraries, and the second layer was to access the pages and provide navigation. The third and top layer created a domain language using fixture-type methods that mapped to the business needs. This allowed the manual testers to write high-level automated tests for workflows before coding was completed. If a fixture didn’t exist because of new functionality, the test could be created and the action word for the missing fixture could be “dummied” in. As soon as the fixture was coded, the test could be run as an acceptance test.

将 Ruby 与 Watir 结合使用的一个非常简单的示例结合了 DSL 的思想。创建的方法是为了简化测试,这样任何测试人员都可以在不了解任何 Ruby 或 Watir 的情况下实际创建一个自动化脚本。

A very simple example of using Ruby with Watir incorporates the idea of DSL. Methods were created to simplify the tests so that any of the testers could actually create an automated script without knowing any Ruby or Watir.

下一个示例显示了一个测试,然后是测试中使用的两种方法。

This next example shows a test, and then two of the methods used in the test.

def test_create_new_user



   login 'administrator','admin'

   navigate_to_tab 'Manage Users'

   click_button "Create New User"

   set_text_field "userFirstNameInput", "Ruby"

   set_text_field "userLastNameInput", "RubyTester"

   click_button "Save Changes"

   verify_text "Saved changes"

end





# methods created to support easier test writing

def navigate_to_tab(menuItemName)

    @browser.link(:text,menuItemName).click

  end



def set_text_field(id, value)

    @browser.text_field(:id,id).set value

  end

def test_create_new_user



   login 'administrator','admin'

   navigate_to_tab 'Manage Users'

   click_button "Create New User"

   set_text_field "userFirstNameInput", "Ruby"

   set_text_field "userLastNameInput", "RubyTester"

   click_button "Save Changes"

   verify_text "Saved changes"

end





# methods created to support easier test writing

def navigate_to_tab(menuItemName)

    @browser.link(:text,menuItemName).click

  end



def set_text_field(id, value)

    @browser.text_field(:id,id).set value

  end

create_new_user如果被多次调用,可以很容易地添加第三层。只需提取测试可以调用的公共代码:

A third level could easily be added if create_new_user was called more than once. Just extract the common code that the test could call:

create_new_user (Ruby, RubyTester)

create_new_user (Ruby, RubyTester)

这些测试非常适合指导开发和提供快速反馈。使测试人员和客户易于编写测试,同时保持为最佳可维护性而设计的自动化框架,降低了测试的总拥有成本。

These tests were well suited to guiding development and providing quick feedback. Making tests easy for testers and customers to write, while keeping the automation framework designed for optimum maintainability, reduced the total cost of ownership of the tests.

——珍妮特

—Janet

您使用的任何工具总是有缺点。例如,使用对象有限制。有时,程序员会使用您的工具可能无法理解的自定义控件或新工具包。

There are always drawbacks to any tool you use. For example, there are limitations to using objects. Sometimes programmers use custom controls or a new toolkit that your tool might not understand.

珍妮特的故事

Janet’s Story

我开始了一份质量保证经理的新工作,经过深思熟虑,我们决定放弃团队已经使用了几年的供应商工具。我们无法弄清楚实际运行的测试是什么,或者真正的覆盖范围是什么。我们决定开始使用 Ruby 和 Watir 来自动化测试。起初自动化进行得相当快,但随后测试开始失败。我们花了很多时间来更改测试以反映新的对象名称。开发人员只是使用默认的 WebLogic 对象名称,每次将新对象添加到页面时,这些名称都会发生变化。测试人员去找开发人员询问他们是否可以改变他们编码的方式。这需要一点说服力,但当开发人员意识到他们的做法造成的问题时,他们改变了习惯。随着时间的推移,所有的默认值都被改变了,每个对象都有一个指定的名称。测试变得更加稳健,我们在维护模式上花费的时间也少了很多。

I started a new job as QA manager, and after much deliberation we decided to drop the vendor tool that the team had been using for a couple of years. We could not figure out what tests were actually being run, or what the real coverage was. We decided to start automating the tests using Ruby and Watir. The automation went fairly quickly at first, but then the tests started failing. We spent a lot of time changing the tests to reflect new object names. The developers were just using the default WebLogic object names, which would change every time a new object was added to the page. The testers went to the developers to ask if they could change the way they were coding. It took a little convincing, but when the developers realized the problems their practice was causing, they changed their habits. Over time, all of the defaults were changed, and each object had an assigned name. The tests became much more robust, and we spent much less time in maintenance mode.

——珍妮特

—Janet

实施新的测试自动化工具通常需要进行一些实验,以在可测试代码和设计良好的测试脚本之间取得良好的平衡。让整个团队参与进来会让这件事变得容易得多。Watir 是我们发现非常适合敏捷项目的 GUI 测试工具的一个示例。让我们再看几个,Selenium 和 Canoo WebTest。

Implementing a new test automation tool usually requires some experimentation to get a good balance of testable code and well-designed test scripts. Involving the whole team makes this much easier. Watir is one example of a GUI test tool that we’ve found is well suited to agile projects. Let’s look at a couple more, Selenium and Canoo WebTest.

硒。Selenium 是另一个开源工具,实际上是一套工具,用于测试 Web 应用程序。测试可以编写为 HTML 表格或用多种流行的编程语言编码,并且可以直接在大多数现代网络浏览器中运行。名为“Selenium IDE”的 Firefox 插件提供了一种快速学习该工具的方法。提供了一个记录器来帮助创建测试,包括编写断言。可以使用多种不同的通用编程和脚本语言编写测试,包括 Java、C# 和 Ruby。

Selenium. Selenium is another open source tool, actually a suite of tools, for testing web applications. The tests can be written as HTML tables or coded in a number of popular programming languages, and can be run directly in most modern web browsers. A Firefox plug-in called “Selenium IDE” provides a way to learn the tool quickly. A recorder is provided to help create the tests, including writing assertions. Tests can be written in several different common programming and scripting languages, including Java, C#, and Ruby.

有关使用 Selenium RC 创建特定领域的测试自动化框架的示例,请参阅第 14 章,“敏捷测试自动化策略”。

See Chapter 14, “An Agile Test Automation Strategy,” for an example of using Selenium RC to create a domain-specific test automation framework.

Canoo 网络测试。在 WebTest 脚本中,测试被指定为 XML 文件中的“步骤”,通过 Web UI 模拟用户的操作。下面是 WebTest 脚本如何调用页面并验证结果的示例:

Canoo WebTest. In WebTest scripts, tests are specified as “steps” in XML files, simulating a user’s actions through a web UI. Here’s an example of how a WebTest script might invoke a page and verify the results:

<setInputField description="set query" name="q" value="Agile Tester"/>

<clickButton description="submit query" label="Google Search"/>

<verifyText description="check for result" text="Lisa Crispin" />

<verifyText description="check for result" text="Janet Gregory" />

<setInputField description="set query" name="q" value="Agile Tester"/>

<clickButton description="submit query" label="Google Search"/>

<verifyText description="check for result" text="Lisa Crispin" />

<verifyText description="check for result" text="Janet Gregory" />

WebTest 不像 Selenium 和 Watir 那样驱动实际的浏览器,而是使用 HtmlUnit 模拟所需的浏览器。与编写测试脚本相比,指定测试的优势在于它们没有逻辑,您不必测试测试。

Rather than driving an actual browser, as Selenium and Watir do, WebTest simulates the desired browser using HtmlUnit. The advantage of specifying tests as opposed to coding test scripts, is because there’s no logic in them, you don’t have to test the test.

丽莎的故事

Lisa’s Story

出于多种原因,我的团队选择 WebTest 来为我们的遗留应用程序自动化冒烟测试。因为脚本是用 XML 编写的,所以团队中的程序员很乐意使用该工具。它使用 Ant 来运行测试,因此将它集成到持续构建过程中很简单。它易于学习,并且测试可以模块化方式设计,因此它们相当容易维护。WebTest 支持测试 PDF 文件、电子邮件和 Excel 文件,所有这些都在我们的应用程序中广泛使用。

My team chose WebTest to automate smoke tests for our legacy application for several reasons. Because the scripts are written in XML, the programmers on the team were comfortable using the tool. It uses Ant to run the tests, so integrating it into the continuous build process was simple. It’s easy to learn, and the tests can be designed in a modular fashion, so they’re fairly easy to maintain. WebTest supports testing PDF files, emails, and Excel files, all of which are widely used in our application.

习惯了强大的商业测试工具,我对指定测试的概念持怀疑态度,而不是对它们进行编程。我对简单测试在捕获回归错误方面的有效性感到惊讶。可以使用 Groovy 或其他脚本语言将逻辑放入测试中,但我们只在少数情况下发现了这种需要。

Being accustomed to powerful commercial test tools, I was skeptical of the concept of specifying tests, as opposed to programming them. I was amazed at how effective the simple tests were at catching regression bugs. It’s possible to put logic into the tests using Groovy or other scripting languages, but we’ve only found the need in a few cases.

每次迭代编写一些测试,我在八个月内对我们应用程序的所有关键区域进行了自动化冒烟测试。这些简单的测试会定期发现回归错误。我们经常重构测试,因此它们相对容易维护。我们在这些测试上的投资回报率非常高。

Writing a few tests per iteration, I automated smoke tests for all of the critical areas of our application in eight months. These simple tests find regression bugs regularly. We refactor the tests frequently, so they are relatively easy to maintain. Our ROI on these tests has been tremendous.

——丽莎

—Lisa

在我们撰写本书时,Selenium、WebTest 和 Watir 只是众多可用于 GUI 测试的开源工具中的三个示例。许多团队编写自己的测试自动化框架。让我们在下一节中看一个示例。

Selenium, WebTest, and Watir are just three examples of the many open source tools available for GUI testing as of the time we wrote this book. Many teams write their own test automation frameworks. Let’s look at an example in the next section.

“自制”测试自动化工具

Bret Pettichord [ 2004 ] 创造了术语“自制”来指代敏捷团队为满足他们自己独特的测试需求而创建的工具。这允许比开源工具更多的定制。这些工具的目标通常是为非技术客户团队成员和测试人员提供一种方式来编写实际上可由自动化工具执行的测试。自制工具是根据项目的确切需求量身定制的。它们的设计可以最大限度地降低总拥有成本。它们通常建立在现有开源工具之上。

Bret Pettichord [2004] coined the term “home-brewed” for the tools agile teams create to meet their own unique testing needs. This allows even more customization than an open source tool. The goal of these tools is usually to provide a way for nontechnical customer team members and testers to write tests that are actually executable by the automated tool. Home-brewed tools are tailored to the exact needs of the project. They can be designed to minimize the total cost of ownership. They’re often built on top of existing open source tools.

Janet 参与了一些使用 Ruby 和 Watir 创建功能测试完整框架的项目。这些框架允许客户指定测试,然后将其转化为功能回归套件。

Janet has been involved in a few projects that have used Ruby and Watir to create a full framework for functional testing. These frameworks allowed customers to specify tests that were then turned into a functional regression suite.

没有测试工具可以保证成功。事实上,测试自动化的历史充满了失败的尝试。让整个团队考虑使用最好的工具是一个很大的帮助,但无论你使用什么工具,你都需要一种聪明的方法来编写测试。我们将在下一节中讨论。

No test tool guarantees success. In fact, the history of test automation is littered with failed attempts. Having the whole team think about the best tools to use is a big help, but no matter what tool you use, you need a smart approach to writing tests. We’ll discuss that in the next section.

编写测试的策略

Strategies for Writing Tests

如果您不明智地使用它们,那么世界上最好的工具也无济于事。测试工具可能使指定测试变得非常容易,但是您是否在正确的时间指定正确的测试取决于您。Lisa 的团队发现,过多的前期细节会使大局蒙上阴影,以至于程序员不知道要编写什么代码。并非每个团队都如此,在某些时候我们确实需要详细信息。提供它们的最晚时间是当程序员拿起编码任务卡并开始编写故事时。

The best tools in the world won’t help if you don’t use them wisely. Test tools might make it very easy to specify tests, but whether you’re specifying the right tests at the right time is up to you. Lisa’s team found that too much detail up front clouded the big picture to such a degree that the programmers didn’t know what to code. This won’t be true for every team, and at some point we do need details. The latest time to provide them is when a programmer picks up a coding task card and starts working on a story.

编写能够有效传达所需行为的详细测试用例需要艺术和科学。表达不当的场景和设计不当的测试用例会造成比解决问题更多的混乱。进行实验,以便您可以为每个故事找到正确的详细程度和正确的测试设计。让我们看看一些策略,以帮助您成功地使用工具来编写有用的面向业务的测试。

Writing detailed test cases that communicate desired behavior effectively requires both art and science. Poorly expressed scenarios and poorly designed test cases can create more confusion than they resolve. Experiment so that you can find the right level of detail and the right test design for each story. Let’s look at some strategies to help you use tools successfully to write useful business-facing tests.

增量构建测试

Build Tests Incrementally

在我们定义了高级验收测试以便程序员知道开始编码的内容之后,我们可以开始详细说明故事测试的其余部分。我们可以与程序员密切合作,以确保我们以最佳方式实现自动化。

After we have defined our high-level acceptance tests so that the programmer knows what to start coding, we can start elaborating on the rest of the story tests. We can work closely with the programmer to ensure we automate the best possible way.

当程序员开始处理故事的编程任务时,请开始编写详细的测试。对于我们这些喜欢测试的人来说,很容易立即去寻找最大的“气味”,即我们认为代码可能脆弱的区域。抵制诱惑。确保最明显的用例首先起作用。编写一个简单、快乐的路径自动化测试,以显示代码完成了它应该完成的最基本的任务。测试通过后,您就可以开始变得更有创意了。编写面向业务的测试是一个迭代过程。

When a programmer starts working on the programming tasks for a story, start writing detailed tests. For those of us who enjoy testing, it’s tempting to go for the biggest “smells” right away, the areas where we think the code might be fragile. Resist the temptation. Make sure the most obvious use case is working first. Write a simple, happy path automated test to show the code accomplishes the most basic task that it should. After that test passes, you can start getting more creative. Writing the business-facing tests is an iterative process.

丽莎的故事

Lisa’s Story

我开始编写支持团队的可执行面向业务的测试,方法是根据产品所有者提供的示例编写一个简单的 FitNesse 测试。我将其展示给编写代码的程序员。他可以立即提出更改建议,或者当他准备好将测试自动化时,他可以根据需要自行修改测试。讨论测试通常会使程序员意识到他错过或误解了需求。我们可能需要与客户进行另一次三方对话。程序员相应地更新代码。我们还可以向产品所有者展示测试,以确保我们正确地捕获了行为。

I start writing executable business-facing tests that support the team by writing a simple FitNesse test based on examples that the product owner provides. I show this to the programmer working on the code. He can make suggestions for changes right then, or he might modify the test himself as appropriate when he’s ready to automate it. Discussing the test often leads the programmer to realize he missed or misunderstood a requirement. We might need another three-way conversation with the customer. The programmer updates the code accordingly. We can also show the test to the product owner to make sure we captured the behavior correctly.

简单的测试通过后,我写更多的测试,覆盖更多的业务规则。我编写一些更复杂的测试,运行它们,程序员根据需要更新代码或测试。这个故事正在填写以提供所有期望的价值。

After the simple test passes, I write more tests, covering more business rules. I write some more complex tests, run them, and the programmer updates the code or tests as needed. The story is filling out to deliver all of the desired value.

——丽莎

—Lisa

第 18 章,“编码和测试”,详细介绍了测试人员和程序员如何协同工作以进行测试和编码。

Chapter 18, “Coding and Testing,” goes into more detail about how testers and programmers work together to test and code.

将每个测试限制在一个业务规则或条件中。在某些时候,您可以自动或手动执行更复杂的场景,但首先要用一个简单的测试来涵盖每个条件。如果您遵循了我们推荐的薄片或钢丝模式,第一组测试应该证明第一端到端的薄片。当您的自动化测试通过时,将它们添加到在频繁构建过程中运行的回归套件中。

Confine each test to one business rule or condition. At some point you can automate or manually perform more complex scenarios, but start by covering each condition with a simple test. If you’ve followed our recommended thin slice or steel thread pattern, the first set of tests should prove the first thin slice end-to-end. As your automated tests pass, add them to the regression suite that runs in a frequent build process.

保持测试通过

Keep the Tests Passing

测试通过后,除非需求发生变化,否则不应失败。如果发生这种情况,则应在更改代码之前更新测试。当然,如果作为需求变更的一部分忘记了测试,我们预计它会失败。它完成了它作为变化检测器的工作。此时,测试可能需要更改才能通过。

After a test passes, it shouldn’t fail unless the requirements were changed. If that happens, the test should be updated before the code is altered. Of course, if a test was forgotten as part of a requirement change, we expect it to fail. It did its job as change detector. At this time, the test will likely need to change to get it passing.

每当持续集成和构建过程中的测试失败时,团队的最高优先级(关键生产问题除外)应该是让构建再次通过。不要注释掉失败的测试并在以后修复它;那是灭亡之路。很快你就会有几十个注释掉的测试和大量的技术债务。团队中的每个人都应该停止他们正在做的事情并确保构建再次“绿色”。确定是否引入了错误,或者是否只需要更新测试以适应有意更改的行为。修复问题,将其签入,并确保所有测试都通过。

Whenever a test fails in a continuous integration and build process, the team’s highest priority (other than a critical production problem) should be to get the build passing again. Don’t comment out the failing test and fix it later; that’s the road to perdition. Soon you’ll have dozens of commented-out tests and a lot of technical debt. Everyone on the team should stop what they’re doing and make sure the build goes “green” again. Determine if a bug has been introduced, or if the test simply needs to be updated to accommodate intentionally changed behavior. Fix the problem, check it in, and make sure all of the tests pass.

丽莎的故事

Lisa’s Story

在我们敏捷工作的早期,我的团队没有足够快地修复损坏的测试。我写了“测试不是暂时的!” 在白板上提醒大家,一旦测试通过,就需要继续通过。几天后,“但是测试人员是!”这句话出现了。被添加来报复我。在那之后,我们确实在保持我们的构建“绿色”方面做得更好。

Early on in our agile efforts, my team wasn’t fixing broken tests fast enough. I wrote “Tests are not temporary!” on the whiteboard to remind everyone that once a test passes, it needs to keep passing. A few days later, the words “but testers are!” had been added to get back at me. We did get much better at keeping our builds “green” after that.

——丽莎

—Lisa

一个通过的测试会导致另一个。通过重构使您的测试保持最新和可维护。扩展它们以涵盖其他测试用例。各种组合和场景在通过后可能会或可能不会成为回归套件的一部分。我们希望我们的回归套件能够及时运行,对边缘情况进行过多测试会减慢速度。

One passing test leads to another. Keep your tests current and maintainable with refactoring. Extend them to cover other test cases. The various combinations and scenarios might or might not become part of the regression suite after they pass. We want our regression suite to run in a timely manner, and having too many tests for edge cases would slow it down.

使用适当的测试设计模式

Use Appropriate Test Design Patterns

在设计测试时,查看不同的模式并选择适合您的模式。使它们尽可能简单。在设计测试之前,您必须确定所需的测试。Pierre Veragen 创造了术语测试起源模式来表示帮助您思考测试的模式。示例和用例提供给我们的测试起源模式。

When designing tests, look at different patterns and choose the ones that work for you. Keep them as simple as you can. Before you can design tests, you have to identify the ones you need. Pierre Veragen coined the term test genesis patterns to note the patterns that help you think of tests. Examples and use cases feed into our test genesis patterns.

建设/运营/检查

Lisa 的团队通常采用构建/操作/检查模式:根据测试目的,在内存中或实际上在数据库中构建输入数据;调用生产代码对这些输入进行操作;并检查该操作的结果。一些团队将此称为设置/执行/验证。例如,要测试提供给新账户持有人的发票,设置要收取的费用,输入与费用金额相关的账户属性,运行计算费用的代码,然后检查费用是多少实际收费。见图9-9举一个测试示例,该测试设置了具有指定金额、利率、期限、付款频率和服务开始日期的贷款,然后检查生成的分期付款时间表。测试数据内置于内存中,有利于快速测试。“拆卸”夹具(未显示)从内存中删除测试数据,因此不会干扰后续测试。

Lisa’s team often goes with a build/operate/check pattern: Build the input data, in memory or actually in the database, depending on the purpose of the test; invoke the production code to operate on those inputs; and check the results of that operation. Some teams call this setup/execute/validate. For example, to test the invoice presented to a new account holder, set up the fees to be charged, input the properties of the account that relate to fee amounts, run the code that calculates the fees, and then check to see what fees were actually charged. See Figure 9-9 for an example of a test that sets up a loan with a specified amount, interest rate, term, payment frequency, and service start date and then checks the resulting amortization schedule. The test data is built in memory, which makes for a speedy test. A “teardown” fixture (not shown) removes the test data from memory so it won’t interfere with subsequent tests.

图 9-9使用构建/操作/检查模式的示例测试

Figure 9-9 Example test with build/operate/check pattern

图像

如果需要测试应用程序的数据访问层,可以使用实际数据库运行测试。每个测试都可以插入自己需要的测试数据,进行操作,检查结果,删除数据。使用真实数据库中的数据进行测试可以成为针对数据访问层和业务逻辑层不易分离的遗留代码进行自动化测试的一种方法。

If there’s a need to test the application’s data access layer, tests can run using an actual database. Each test can insert the test data it needs, operate on it, check results, and delete the data. Testing with data in a real database can be a means of automating a test against legacy code whose data access and business logic layers aren’t easily separated.

请注意,示例中的“check”表使用声明式样式,每一行构成一个独立的测试用例,不会改变系统的状态。我们示例中的每一行都测试贷款摊销计划中的一行。在下一节中,我们将研究过程风格的模式,其中包含更改或测试系统状态的步骤。

Notice that the “check” table in the example uses a declarative style, with each row forming an independent test case, without changing the state of the system. Each row in our example tests a line in the loan amortization schedule. In the next section, we’ll look at patterns that are in a procedural style, with steps that change or test the state of the system.

基于时间、活动和事件模式

有时,基于时间线的程序模式可以更好地反映业务。例如,在测试贷款时,我们希望确保每笔付款都正确应用了利息和本金。利息金额取决于收到付款的日期和最后处理付款的日期。我们想要一个模拟以一定的金额、利率和时间段贷款的测试,然后随着时间的推移模拟借款人发送付款,这些付款会被接收和处理。图 9-10显示了 FitLibrary“DoFixture”测试的一个简单示例,该测试提取贷款、检查付款金额、发布借款人的付款、接收付款并处理它们,然后检查利息、本金和贷款余额. 它还检查贷款违约状态。

Sometimes a timeline-based procedural pattern reflects the business better. For example, when testing a loan, we want to make sure interest and principal are applied correctly for each payment. The amount of interest depends on the date the payment was received and the date of the last payment processed. We want a test that simulates taking out a loan for a certain dollar amount, interest rate, and time period, and then over time simulates the borrower sending in payments, which are received and processed. Figure 9-10 shows a simple example of a FitLibrary “DoFixture” test that takes out a loan, checks the payment amount, posts the borrower’s payments, receives the payments and processes them, and then checks the interest, principal, and loan balance amount. It also checks the loan default state.

图 9-10基于时间的测试示例

Figure 9-10 Sample time-based test

图像

根据领域的不同,基于时间或事件的方法可能比声明式类型测试更好地模拟实际业务流程并且更容易被业务专家理解。其他客户可能会发现声明式表格样式更易于理解,因为它隐藏了程序细节。不同的模式最适合不同的情况,因此请尝试使用它们。

Depending on the domain, a time- or event-based approach might simulate the actual business processes better and be more understandable to business experts than a declarative type test. Other customers might find the declarative table style simpler to understand, because it hides the procedural details. Different patterns work best for different situations, so experiment with them.

学习更多

您的团队应该自学有助于推动编程的测试模式。为每种类型的测试找到正确的模式可确保测试传达清晰、易于维护并在最佳时间内运行。有关测试设计的更多宝贵资源,请参阅参考书目,例如 Gerard Meszaros 的xUnit 测试模式:重构测试代码

Your team should educate itself on test patterns that help drive programming. Finding the right pattern for each type of test ensures the test communicates clearly, is easy to maintain, and runs in an optimal amount of time. See the bibliography for more invaluable resources on test design, such as Gerard Meszaros’s xUnit Test Patterns: Refactoring Test Code.

将程序员和测试人员聚集在一起,集思广益测试方法,并帮助决定哪些测试可以自动化以及代码应该如何设计以支持测试。业务逻辑和算法应该可以通过测试装置访问,而无需通过用户界面或批处理调度过程。这使得测试驱动开发成为可能,进而产生可测试的架构。

Bring programmers and testers together to brainstorm test approaches and to help decide what tests can be automated and how the code should be designed to support testing. Business logic and algorithms should be accessible by test fixtures, without having to go through a user interface or batch scheduling process. This enables test-driven development, which in turn produces testable architecture.

自动化测试的一种常见方法是使用关键字或动作词来驱动测试。这可以与 Fit 和 FitNesse 等工具一起使用,或者与 Watir 一起使用 Ruby。我们接下来会解释这个。

A common approach to automating tests is by driving tests with keywords or action words. This can be used with tools such as Fit and FitNesse, or Ruby with Watir. We’ll explain this next.

关键字和数据驱动测试

Keyword and Data-Driven Tests

数据驱动测试是一种有助于减少测试维护并使您能够与手动测试人员共享测试自动化的工具。很多时候您想要一遍又一遍地运行相同的测试代码,只重复输入和预期结果。电子表格或表格(例如 Fit 支持的表格)是指定输入的绝佳方式。测试夹具、方法或脚本可以一次循环遍历每个数据值,将预期结果与实际结果相匹配。通过使用数据驱动测试,您实际上是在使用示例来展示应用程序应该做什么。

Data-driven testing is a tool that can help reduce test maintenance and enable you to share your test automation with manual testers. There are many times when you want to run the same test code over and over, repeating only the inputs and expected results. Spreadsheets or tables, such as those supported by Fit, are excellent ways to specify inputs. The test fixture, method, or script can loop through each data value one at a time, matching expected results to actual results. By using data-driven tests, you are actually using examples to show what the application is supposed to do.

关键字驱动测试是自动化测试中使用的另一种工具,其中使用预定义的关键字来定义操作。这些动作对应于与应用程序相关的进程。这是创建域测试语言的第一步。这些关键字(或动作词)代表一种非程序员可以用来开发自动化测试的非常简单的规范语言。您仍然需要程序员或技术自动化专家来实现动作词所作用的固定装置。如果扩展这些关键字以模拟领域语言,客户和非技术测试人员可以更轻松地指定映射到工作流的测试。

Keyword-driven testing is another tool used in automated testing, where predefined keywords are used to define actions. These actions correspond to a process related to the application. It is the first step in creating a domain testing language. These keywords (or action words) represent a very simple specification language that non-programmers can use to develop automated tests. You still need programmers or technical automation specialists to implement the fixtures that the action words act on. If these keywords are extended to emulate the domain language, customers and nontechnical testers can specify tests that map to the workflow more easily.

图 9-11中的示例电子表格显示了一家公司如何使用动作词来自动化他们的测试设置。可以使用相同的动作词进行测试。Signup、Signoff 和 CCDeposit 是特定于域的词。他们的用户可以在不了解底层代码的情况下轻松编写测试。

The sample spreadsheet in Figure 9-11 shows how one company used action words to automate their test setup. The same action words can be used to test. The words Signup, Signoff, and CCDeposit are words that are domain-specific. Their users could easily write tests without understanding the underlying code.

图片 9-11带有动作词的示例测试电子表格

Figure 9-11 Sample test spreadsheet with action words

图像

结合数据驱动和关键字驱动的测试技术会非常强大。Fit 和 FitNesse 同时使用关键字和数据来驱动测试。我们在本章中描述的其他工具也可以采用这种方法。

Combining data-driven and keyword-driven testing techniques can be very powerful. Fit and FitNesse use both keywords and data to drive tests. The other tools we’ve described in this chapter can also accommodate this approach.

如果代码不是为易于测试而设计的,那么任何测试策略都可能会遇到麻烦。让我们来看看可测试性问题。

Any test strategy can run into trouble if the code isn’t designed to be easily tested. Let’s take a look at testability concerns.

可测试性

Testability

使用适当的设计模式构建并在任何编码之前编写的面向业务的测试可帮助团队实现可测试的代码设计。程序员从查看面向业务的测试开始,也许与测试人员、分析师或客户一起,以便在他们继续进行测试驱动设计时,执行这些测试的需求始终在他们的脑海中。他们可以构建,以便测试提供输入并控制运行时条件。

Business-facing tests built with appropriate design patterns and written ahead of any coding help the team achieve a testable code design. The programmers start by looking at the business-facing tests, perhaps together with a tester, analyst, or customer, so that the need to execute those tests are always in their minds as they proceed with their test-driven design. They can build so that the tests provide inputs and control run-time conditions.

珍妮特的故事

Janet’s Story

当我尝试使用 Ruby 和 Watir 自动化一些 GUI 工作流时,我遇到了一个障碍。无法识别日历弹出功能,并且数据字段是只读的。我把我的问题交给了一位程序员。我们结对在一起,这样他就能看到我遇到的问题。他做的第一件事就是了解日历功能。他认为自动化测试太难了,所以他提出了另一种选择。他创建了一种新方法,可以“愚弄”输入字段,使其接受文本字段中的日期。我们知道风险在于日历上没有自动化,但为了简单起见,我们采用了他的选择。

I ran into a snag when I was trying to automate some GUI workflow with Ruby and Watir. The calendar pop-up feature was not recognized, and the data field was read-only. I took my problem to one of the programmers. We paired together so that he could see the issue I was having. The first thing he did was to understand the calendar feature. He thought it would be too difficult to automate the test, so he suggested another alternative. He created a new method that would “fool” the input field so it would accept a date into the text field. We knew the risk was no automation on the calendar, but for simplicity’s sake we went with his option.

并非所有代码都可以使用自动化进行测试,但与程序员一起寻找问题的替代解决方案。

Not all code is testable using automation, but work with the programmers to find alternative solutions to your problems.

——珍妮特

—Janet

让我们看看促进可测试代码设计的技术。

Let’s look at techniques that promote design of testable code.

代码设计和测试设计

Code Design and Test Design

第 7 章支持团队的面向技术的测试”中,我们解释了单元级别的测试驱动开发如何确保可测试的架构。面向业务的测试也是如此。Lisa 团队设计的分层架构同样适用于功能测试。可以直接针对业务逻辑进行测试,而无需涉及用户界面,如果合适,也无需涉及数据库层。这并不意味着不需要测试数据库层。它仍然需要测试,也许在其他地方。

In Chapter 7, “Technology-Facing Tests that Support the Team,” we explained how test-driven development at the unit level ensures a testable architecture. This is true for business-facing tests as well. The layered architecture Lisa’s team designed works just as well for functional testing. Testing can be done directly against the business logic without involving the user interface, and if appropriate, without involving the database layer. This doesn’t mean that the database layer doesn’t need to be tested. It still needs to be tested, just maybe somewhere else.

在对表示层进行编码时也必须考虑可测试性。GUI 测试工具在使用良好实践开发的设计良好的代码上工作得更好。

Testability has to be considered when coding the presentation layer as well. GUI test tools work better on well-designed code developed with good practices.

丽莎的故事

Lisa’s Story

当我第一次开始尝试使用 Canoo WebTest 进行自动化 GUI 测试时,我发现系统中使用的 HTML 和 JavaScript 不符合标准并且包含许多错误。WebTest 及其构建的工具 HtmlUnit 需要正确的标准 HTML 和 Javascript。指定测试取决于良好的 HTML实践,例如给每个元素一个唯一的 ID。程序员开始编写 HTML 和 JavaScript(以及后来的 Ajax)时考虑到测试工具,使测试自动化变得更加容易。他们还开始验证他们的 HTML 并确保它符合行业标准。这也减少了应用程序在不同浏览器和浏览器版本中出现问题的可能性。

When I first started trying to automate GUI tests using Canoo WebTest, I discovered that the HTML and JavaScript used in the system didn’t comply with standards and contained many errors. WebTest and the tool it’s built on, HtmlUnit, required correct, standard HTML and Javascript. Specifying tests depended on good HTML practices such as giving each element a unique ID. The programmers started writing HTML and JavaScript (and later, Ajax) with the test tool in mind, making test automation much easier. They also started validating their HTML and making sure it was up to industry standards. This also reduced the possibility of the application having problems in different browsers and browser versions.

——丽莎

—Lisa

编码和测试是敏捷开发过程中的一部分。代码设计和测试设计是相辅相成、相互依存的。这是一个先有鸡还是先有蛋的场景:没有可测试的代码设计就无法编写测试,而没有设计良好的测试(明确传达需求并与系统架构兼容)就无法编写代码。这就是为什么我们总是将编码和测试放在一起考虑。当我们估算故事时,我们包括编码和测试的时间,当我们计划每个迭代和故事时,我们预算时间来设计测试和代码。如果证明自动化测试很困难,请评估代码设计。如果程序员编写的代码不符合客户的期望,问题可能出在测试设计不当。

Coding and testing are part of one process in agile development. Code design and test design are complementary and interdependent. It’s a chicken-andegg scenario: You can’t write tests without a testable code design, and you can’t write code without well-designed tests that clearly communicate requirements and are compatible with the system architecture. This is why we always consider coding and testing together. When we estimate stories, we include time for both coding and testing, and when we plan each iteration and story, we budget time to design both tests and code. If automating a test proves difficult, evaluate the code design. If programmers are writing code that doesn’t match customer expectations, the problem might be poorly designed tests.

自动化与手动象限 2 测试

Automated vs. Manual Quadrant 2 Tests

我们假设至少有相当一部分指导编程的测试是自动化的。如果您及早与程序员共享手动测试场景,它们也可以推动编程。越早将它们转化为自动化测试,您就会越快地实现收益。大多数手动测试更多地属于“评论产品”象限,在那里我们可能会了解我们在最初的一组测试中没有预料到的故事。

We’ve assumed that at least a good-sized portion of the tests that guide programming will be automated. Manual test scenarios can also drive programming if you share them with the programmers early. The earlier you turn them into automated tests, the faster you will realize the benefit. Most manual tests fall more into the “critique product” quadrant where we might learn things about the story we hadn’t anticipated with the initial set of tests.

第 IV 部分测试自动化”中,我们将深入探讨如何开发成功的测试自动化策略,并研究诸如构建您自己的工具与使用第三方或开源工具之类的考虑因素。

In Part IV, “Test Automation,” we’ll dive into developing a successful test automation strategy and look at considerations such as building your own tools versus using third-party or open source tools.

这并不能阻止我们编写可能不适合自动化的测试。编写测试时不要担心细节。您可能会想出一次性测试,这些测试很重要,但在回归套件中一遍又一遍地重复并不重要。您可能会开始考虑端到端的场景或探索性测试会话的跳板,这些会话可能会通过一些自动化来促进,但需要一个聪明的人来全面执行它们。你稍后会明白的。现在,我们要确保我们捕捉到了客户的关键需求。

That doesn’t stop us from writing tests that might not be appropriate for automation. Don’t sweat the details when you’re writing tests. You might come up with one-off tests that are important to do but not important to repeat over and over in a regression suite. You might start thinking about end-to-end scenarios or springboards to exploratory test sessions that might be facilitated with some automation but need an intelligent human to conduct them in full. You’ll figure that out later. Right now, we want to make sure we capture the customer’s critical requirements.

从一个简单的方法开始,看看它是如何工作的,并以此为基础进行构建。重要的是开始编写面向业务的测试以在开发产品时支持团队。

Start with a simple approach, see how it works, and build on it. The important thing is to get going writing business-facing tests to support the team as you develop your product.

测试管理

Test Management

如果我们要自动化测试,将它们呈现在自动化工具框架中是有意义的,即使它们还不可执行。我们希望所有测试(即使是那些不会自动进行的测试)都可以为开发团队中的每个人访问并为我们的客户所理解。有很多选项可以让团队中的每个人都看到测试。Wiki 是共享测试用例的常用方式,一些工具(如 FitNesse)使用 wiki 或类似工具,使叙述性需求、示例和可执行测试共存于一个地方。

If we’re automating tests, it makes sense to present them in the automation tool framework, even if they’re not yet executable. We want some way for all tests, even those that won’t be automated, to be accessible to everyone on the development team and understandable to our customers. There are lots of options available that let everyone on the team see tests. Wikis are a common way to share test cases, and some tools such as FitNesse use a wiki or similar tool, enabling narrative requirements, examples, and executable tests to coexist in one place.

第 14 章,“敏捷测试自动化策略”,详细介绍了如何管理自动化测试。

Chapter 14, “An Agile Test Automation Strategy,” goes into more detail on how to manage automated tests.

测试应包含在您的源代码控制中,以便您可以跟踪哪些版本的测试与哪些版本的代码一致。至少,对您的测试进行某种版本控制。一些团队使用可能与需求管理、缺陷跟踪或其他组件集成的测试管理工具或综合测试框架。

Tests should be included in your source code control, so that you can track which versions of the tests go with which versions of the code. At the very least, have some kind of version control for your tests. Some teams use test management tools or comprehensive test frameworks that might integrate with requirements management, defect tracking, or other components.

概括

Summary

在本章中,我们了解了您可能希望在工具包中使用的工具来帮助创建面向业务的测试,以帮助推动开发和指南,以确保这些工具提供帮助而不是阻碍。这些工具和指南包括以下内容:

In this chapter, we’ve looked at tools you might want in your toolkit to help create business-facing tests that help drive development and guidelines to make sure the tools help rather than get in the way. The tools and guidelines included the following:

图像团队需要正确的工具来引出需求和示例,从全局到细节,包括清单、思维导图、电子表格、模型、流程图和各种基于软件的工具。

Teams need the right tools to elicit requirements and examples, from the big picture down to details, including checklists, mind maps, spreadsheets, mock-ups, flow diagrams, and various software-based tools.

图像在 GUI 下方和通过 GUI 表达示例和自动化测试的工具对于敏捷测试自动化也是必不可少的。其中一些工具包括单元测试工具、行为驱动开发工具、FitNesse、Ruby with Watir、Selenium 和 Canoo WebTest。

Tools to express examples and automate tests, below and through the GUI, are also essential to agile test automation. Some of these tools include unit test tools, behavior-driven development tools, FitNesse, Ruby with Watir, Selenium, and Canoo WebTest.

图像“自制”测试自动化可帮助团队将自动化测试的总拥有成本保持在较低水平。

“Home brewed” test automation helps teams keep the total cost of ownership of their automated tests low.

图像通过面向业务的测试推动开发是激励敏捷团队设计可测试代码的一种方式。

Driving development with business-facing tests is one way agile teams are motivated to design testable code.

图像构建自动化的测试策略应该包括逐步构建测试并确保它们始终通过。设计模式可用于帮助您创建有效的测试。

Test strategies for building your automation should include building your tests incrementally and making sure they always pass. Design patterns can be used to help you create effective tests.

图像 关键字和数据驱动的测试是与我们在本章中讨论的工具一起使用的常用方法。

Keyword and data-driven testing is a common approach that works with the tools we’ve discussed in this chapter.

图像在您的代码设计中考虑可测试性,并明智地选择您的测试工具,因为它们需要与您的代码一起工作。

Consider testability in your code design, and choose your test tools wisely, because they need to work with your code.

图像我们需要一些方法来组织测试,以便可以有效地使用它们并将其置于版本控制中。

We need some way to organize tests so that they can be used effectively and put into version control.

第 10 章评价产品的面向业务的测试

Chapter 10 Business-Facing Tests that Critique the Product

图像

本章涵盖测试矩阵的第三象限。在第 8 章支持团队的面向业务的测试”中,我们讨论了第二象限以及如何使用面向业务的测试来支持编程。在本章中,我们将向您展示如何通过不同类型的面向业务的测试来评价产品。我们还将讨论可能有助于这些活动的工具。

This chapter covers the third quadrant of the testing matrix. In Chapter 8, “Business-Facing Tests that Support the Team,” we talked about the second quadrant and how to use business-facing tests to support programming. In this chapter, we show you how to critique the product with different types of business-facing tests. We’ll also talk about tools that might help with these activities.

第三象限简介

Introduction to Quadrant 3

请记住,面向业务的测试是您可以用业务专家会(或应该)感兴趣的术语来描述的测试。当我们提到传统分阶段方法中的测试时,它几乎总是意味着在产品构建后对其进行评判。到目前为止,您可能认为在敏捷开发中这部分测试应该很容易。毕竟,我们只是花了所有时间来确保它按预期工作。这些需求在构建时都经过了测试,包括安全性和其他非功能性需求,对吧?剩下的就是可能找到一些晦涩或有趣的错误。

Remember that business-facing tests are those you could describe in terms that would (or should) be of interest to a business expert. When we mention testing in traditional phased approaches, it pretty much always means critiquing the product after it is built. By now, you might think that in agile development this part of testing should be easy. After all, we just spent all that time making sure it works as expected. The requirements have all been tested as they were built, including security and other nonfunctional requirements, right? All that’s left is to possibly find some obscure or interesting bugs.

作为测试人员,我们知道人们会犯错误。无论我们多么努力地在第一次做对,我们有时还是会做错。也许我们使用了一个没有测试我们认为它做了什么的例子。或者,也许我们记录了一个错误的预期结果,所以测试通过了,但它是一个误报。业务专家可能忘记了真实用户需要的一些东西。最好的客户在看到之前可能不知道自己想要(或不想要)什么。

As testers, we know that people make mistakes. No matter how hard we try to get it right the first time, we sometimes get it wrong. Maybe we used an example that didn’t test what we thought it did. Or maybe we recorded a wrong expected result so the test passed, but it was a false positive. The business expert might have forgotten some things that real users needed. The best customer may not know what she wants (or doesn’t want) until she sees it.

批评或评估产品是测试人员或业务用户在评估和判断产品时所做的事情。这些评估者根据他们是否喜欢新屏幕的行为方式、外观和感觉或工作流程来形成看法。与想象产品向您描述时的样子相比,看到、感觉和触摸产品并做出反应要容易得多。

Critiquing or evaluating the product is what testers or business users do when they assess and make judgments about the product. These evaluators form perceptions based on whether they like the way it behaves, the look and feel, or the workflow of new screens. It is easier to see, feel, and touch a product and respond than to imagine what it will look like when it is described to you.

很难将批评产品的面向业务的测试自动化,因为此类测试依赖于人类的智慧、经验和直觉。然而,自动化工具可以协助象限 3 测试的各个方面(见图10-1),例如测试数据设置。本章的最后一节包含帮助团队专注于评估产品价值的重要方面的工具类型的示例。

It’s difficult to automate business-facing tests that critique the product, because such testing relies on human intellect, experience, and instinct. However, automated tools can assist with aspects of Quadrant 3 tests (see Figure 10-1), such as test data setup. The last section of this chapter contains examples of the types of tools that help teams focus on the important aspects of evaluating the product’s value.

图 10-1象限 3 测试

Figure 10-1 Quadrant 3 tests

图像

虽然我们在本章中讨论的大部分测试都是手动的,但不要错误地认为这种手动测试足以生产出高质量的软件,并且您可以不自动化回归测试而侥幸逃脱。如果你没有在象限 1 和 2 中自动化测试,你将没有时间进行任何象限 3 测试。

While much of the testing we discuss in this chapter is manual, don’t make the mistake of thinking that this manual testing will be enough to produce high-quality software and that you can get away with not automating your regression tests. You won’t have time to do any Quadrant 3 tests if you haven’t automated the tests in Quadrants 1 and 2.

评估或批评产品是关于操纵被测系统并尝试重现最终用户的实际体验。了解不同的业务场景和工作流程有助于使体验更加真实。

Evaluating or critiquing the product is about manipulating the system under test and trying to recreate actual experiences of the end users. Understanding different business scenarios and workflows helps to make the experience more realistic.

示范

Demonstrations

我们建议尽早并经常向客户展示您正在开发的产品。一旦在故事开发过程中有基本的 UI 或报告可用,请将其展示给产品所有者或团队中的其他领域专家。然而,在迭代演示之前,并非业务方面的每个人都有机会看到迭代的可交付成果。迭代结束演示为业务用户和领域专家提供了一个机会,可以查看迭代中交付的内容并修改他们的优先级。这让他们有机会说:“这就是我所说的,但这不是我的意思。” 这是批评产品的一种形式。

We recommend showing customers what you’re developing early and often. As soon as a rudimentary UI or report is available during story development, show it to the product owner or other domain expert on the team. However, not everyone on the business side will get a chance to see the iteration’s deliverables until the iteration demo. End-of-iteration demonstrations are an opportunity for the business users and domain experts to see what has been delivered in the iteration and revise their priorities. It gives them a chance to say, “That’s what I said, but it’s not what I meant.” This is a form of critiquing the product.

珍妮特的故事

Janet’s Story

我从事的项目有五个独立的团队,每组八到十名成员,他们都在开发同一个系统。即使他们在同一楼层,沟通也是一个问题。有很多依赖和重叠,所以程序员依靠团队领导会议来共享信息。但是,业务用户和测试人员需要查看其他团队正在开发的内容。他们依靠每个团队提供的迭代结束演示来了解其他团队在做什么。

I worked on a project that had five separate teams of eight to ten members, all developing the same system. Even though they were on the same floor, communication was an issue. There were many dependencies and overlaps, so the programmers depended on team-lead meetings to share information. However, the business users and testers needed to see what was being developed by other teams. They relied on end-of-iteration demonstrations given by each team to learn what the other teams were doing.

——珍妮特

—Janet

向高管或高层管理人员进行演示也可以增强您对项目的信心。分阶段项目的缺点之一是直到最后才看到任何东西,管理层必须完全信任开发团队的报告。敏捷开发的增量和迭代特性使您有机会在生产时展示业务价值,甚至在发布之前。如果参与者积极询问有关新功能的问题,现场演示可能是一个非常强大的工具。

Demonstrations to the executives or upper management can instill confidence in your project as well. One of the downfalls of a phased project is there is nothing to see until the very end, and management has to place all of its trust in the development team’s reports. The incremental and iterative nature of agile development gives you a chance to demonstrate business value as you produce it, even before you release it. A live demonstration can be a very powerful tool if the participants are actively asking questions about the new features.

与其等到迭代结束,不如利用任何机会展示您的更改。Janet 最近参与的一个项目使用与业务用户的定期会议来演示新功能,以便获得即时反馈。任何需要的更改都被输入到下一次迭代中。

Rather than waiting until the end of the iteration, you can use any opportunity to demonstrate your changes. A recent project Janet worked on used regularly scheduled meetings with the business users to demonstrate new features in order to get immediate feedback. Any desired changes were fed into the next iteration.

选择适合您团队的演示频率,以便反馈循环足够快,以便您将更改合并到发布中。

Choose a frequency for your demonstrations that works for your team so that the feedback loop is quick enough for you to incorporate changes into the release.

第 19 章,“结束迭代”,讨论迭代结束时的演示和回顾。

Chapter 19, “Wrap Up the Iteration,” talks about end-of-iteration demonstrations and reviews.

非正式演示可能会更有成效。与业务专家坐下来,向她展示您的团队目前正在编写的故事。一起做一些探索性测试。我们听说过一些团队让他们的利益相关者在每次迭代演示后进行一些探索性测试,以帮助他们思考改进和未来的故事,以改变或构建刚刚交付的功能。

Informal demos can be even more productive. Sit down with a business expert and show her the story your team is currently coding. Do some exploratory testing together. We’ve heard of teams that get their stakeholders to do some exploratory testing after each iteration demo in order to help them think of refinements and future stories to change or build on the functionality just delivered.

场景测试

Scenario Testing

业务用户可以帮助定义可以模仿最终用户行为的合理场景和工作流。现实生活中的领域知识对于创建准确的场景至关重要。我们想从头到尾测试系统,但不一定是黑盒子。

Business users can help define plausible scenarios and workflows that can mimic end user behavior. Real-life domain knowledge is critical to creating accurate scenarios. We want to test the system from end to end but not necessarily as a black box.

帮助团队了解业务和用户需求的一项好技术是“肥皂剧测试”,这是 Hans Buwalda [2004] 创造的一个术语。这里的想法是采用基于现实生活的场景,以类似于电视肥皂剧夸大行为和情绪的方式对其进行夸大,并将其压缩为一系列快速事件。想想这样的问题,“可能发生的最糟糕的事情是什么,它是如何发生的?”

One good technique for helping the team understand the business and user needs is “soap opera testing,” a term coined by Hans Buwalda [2004]. The idea here is to take a scenario that is based on real life, exaggerate it in a manner similar to the way TV soap operas exaggerate behavior and emotions, and compress it into a quick sequence of events. Think about questions like, “What’s the worst thing that can happen, and how did it happen?”

作为测试人员,我们经常“编造”测试数据,但它通常很简单,因此我们可以很容易地检查我们的结果。在测试不同的场景时,数据和流程都需要符合实际。查明数据是来自另一个系统还是手动输入的。如果可以的话,通过要求客户提供测试数据来获取样品。真实数据将流经系统,并可以沿途进行检查。在大型系统中,它的行为会根据所做的决定而有所不同。

As a tester, we often “make up” test data, but it is usually simple so we can easily check our results. When testing different scenarios, both the data and the flow need to be realistic. Find out if the data comes from another system or if it’s input manually. Get a sample if you can by asking the customers to provide data for testing. Real data will flow through the system and can be checked along the way. In large systems, it will behave differently depending on what decisions are made.

第 14 章,“敏捷测试自动化策略”,检查了获取测试数据的不同方法。

Chapter 14, “An Agile Test Automation Strategy,” examines different approaches to obtaining test data.

帮助定义场景和工作流程的工具可以很简单。数据流或过程流程图将有助于识别一些常见的场景。如果您花时间,这些场景可以帮助您思考复杂的问题。考虑用户及其动机。

Tools to help define the scenarios and workflows can be simple. Data flow or process flow diagrams will help identify some of the common scenarios. These scenarios can help you think through a complex problem if you take the time. Consider the users and their motivation.

丽莎的故事

Lisa’s Story

我们的团队计划重写处理共同基金日常买卖的应用程序的核心功能。这些交易是退休计划参与者做出贡献、将余额从一个基金交换到另一个基金或从他们的账户中提取资金的结果。Lisa 的同事 Mike Thomas 研究了现有的交易处理流程并绘制了图表,以便团队在尝试重写代码之前能够很好地理解它。图 10-2显示了流程图的一部分。WT 代表进行实际交易的托管人。下载三种不同的文件类型并将其转换为可读格式:CFM、PRI 和 POS。这些文件中的每一个都输入到应用程序的不同部分以执行处理并产生各种输出:结算交易、代码异常报告和资金头寸报告。

Our team planned to rewrite the core functionality of the application that processes the daily buys and sells of mutual funds. These trades are the result of retirement plan participants making contributions, exchanging balances from one fund to another, or withdrawing money from their accounts. Lisa’s coworker, Mike Thomas, studied the existing trade processing flow and diagrammed it so that the team could understand it well before trying to rewrite the code. Figure 10-2 shows a portion of the flow diagram. WT stands for the custodian who does the actual trading. Three different file types are downloaded and translated into readable format: CFM, PRI, and POS. Each of these files feeds into a different part of the application to perform processing and produce various outputs: settled trades, a ticker exception report, and a fund position report.

——丽莎

—Lisa

图片 10-2过程流程图的示例部分

Figure 10-2 Sample portion of a process flow diagram

图像

端到端测试时,进行抽查以确保数据、状态标志、计算等按预期运行。使用流程图和其他视觉辅助工具来帮助您理解功能。许多组织依靠报告来做出决定,而这些报告似乎是我们核实的最后一件事。如果您的场景已被正确识别,您可能能够使用您的应用程序报告提供最终检查。

When testing end-to-end, make spot checks to make sure the data, status flags, calculations, and so on are behaving as expected. Use flow diagrams and other visual aids to help you understand the functionality. Many organizations depend on reports to make decisions, and those reports seem to be the last thing we verify. If your scenarios have been identified correctly, you might be able to use your application reports to provide a final check.

探索性测试

Exploratory Testing

探索性测试 (ET) 是敏捷世界中测试的重要方法。作为调查工具,它是对故事测试和我们的自动回归套件的重要补充。这是一种无需脚本即可进行测试的复杂、深思熟虑的方法,它使您能够超越已经测试过的明显变化。探索性测试将学习、测试设计和测试执行结合到一种测试方法中。我们以一种有纪律的方式应用启发式方法和技术,以便“做”揭示出比仅仅思考问题更多的含义。在测试时,您可以了解有关被测系统的更多信息,并可以使用该信息来帮助设计新测试。

Exploratory testing (ET) is an important approach to testing in the agile world. As an investigative tool, it’s a critical supplement to the story tests and our automated regression suite. It is a sophisticated, thoughtful approach to testing without a script, and it enables you to go beyond the obvious variations that have already been tested. Exploratory testing combines learning, test design, and test execution into one test approach. We apply heuristics and techniques in a disciplined way so that the “doing” reveals more implications that just thinking about a problem. As you test, you learn more about the system under test and can use that information to help design new tests.

探索性测试不是通过详尽测试来评估软件的一种方法。它旨在为您的测试添加另一个维度。您所做的只是足以查看“已完成”的故事是否真的让您满意。

Exploratory testing is not a means of evaluating the software through exhaustive testing. It is meant to add another dimension to your testing. You do just enough to see if the “done” stories are really done to your satisfaction.

参考书目列出了您应该调查的更多资源,以了解有关探索性测试的更多信息。

The bibliography lists more resources you should investigate to learn more about exploratory testing.

探索性测试的一个有价值的副作用是由此产生的学习。它揭示了可以使用更多自动化测试的产品领域,并提出了新功能或修改功能的想法,从而产生了新的故事。

A valuable side effect of exploratory testing is the learning that comes out of it. It reveals areas of the product that could use more automated tests and brings up ideas for new or modified features that lead to new stories.

请参阅参考书目以获取有关快速软件测试的更多信息的一些链接。

See the bibliography for some links to more about Rapid Software Testing.

不熟悉探索性测试的人经常将其与临时测试混淆。探索性测试不是坐在键盘前打字。不熟练的“黑盒”测试人员可能不知道如何进行探索性测试。

People unfamiliar with exploratory testing often confuse it with ad hoc testing. Exploratory testing isn’t sitting down at a keyboard and typing away. Unskilled “black box” testers may not know how to do exploratory testing.

探索性测试从一个章程开始,该章程说明将探索功能的哪些方面。它需要批判性思维、解释结果并将其与预期或类似系统进行比较。测试时跟随“气味”是一个重要组成部分。测试人员在探索性测试期间做笔记,以便他们可以重现他们看到的任何问题并根据需要进行更多调查。

Exploratory testing starts with a charter of what aspects of the functionality will be explored. It requires critical thinking, interpreting the results, and comparing them to expectations or similar systems. Following “smells” when testing is an important component. Testers take notes during their exploratory testing sessions so that they can reproduce any issues they see and do more investigation as needed.

探索性测试一词因测试的“上下文驱动学派”而流行。这是一项纪律严明的活动,可以学习。基于会话的测试管理是一种测试方法,旨在使探索性测试可审计和可测量 [ Bach,2003 年]。

The term exploratory testing was popularized by the “context-driven school” of testing. It’s a highly disciplined activity, and it can be learned. Session-based test management is one method of testing that’s designed to make exploratory testing auditable and measurable [Bach, 2003].

基于会话的测试

Session-Based Testing

基于会话的测试结合了问责制和探索性测试。它为测试人员的探索性测试体验提供了一个框架,以便他们能够以一致的方式报告结果。

Session-based testing combines accountability and exploratory testing. It gives a framework to a tester’s exploratory testing experience so that they can report results in a consistent way.

珍妮特的故事

Janet’s Story

James Bach [ 2003 ] 将探索性测试比作拼图。当我第一次阅读他用拼图类比的文章时,探索性测试对我来说非常有意义。

James Bach [2003] compares exploratory testing to putting together a jigsaw puzzle. When I first read his article with the jigsaw puzzle analogy, exploratory testing made perfect sense to me.

我通过倾倒拼图的所有部分然后将它们分类成不同的颜色和边缘部分来开始拼图游戏。接下来,我将边缘部分放在一起,这为我提供了一个开始的框架。拼图的边缘既类似于帮助我集中注意力的使命宣言,也类似于会议的时间限制,它使我保持在一定的范围内。

I start a jigsaw puzzle by dumping out all of the pieces of the puzzle and then sorting them into the different colors and edge pieces. Next, I put the edge pieces together, which gives me a framework in which to start. The edge of the jigsaw is analogous both to the mission statement, which helps me focus, and to the time-boxing of a session, which keeps me within certain limits.

基于会话的测试是探索性测试的一种形式,但它是有时间限制的并且更加结构化。我从 Jonathan Bach 那里了解了基于会话的测试,并发现它为我提供了进行探索性测试所需的结构。我使用与拼图游戏相同的技巧:我寻找颜色或形状的图案,或者寻找看起来不正确的东西,一种异常现象。我的思维过程可以采用这些模式并理解它们,使用我开发的启发式方法来帮助我解决难题。

Session-based testing is a form of exploratory testing, but it is time-boxed and a bit more structured. I learned about session-based testing from Jonathan Bach and found it gave me the structure I needed to do exploratory testing well. I use the same skills as I do for a jigsaw puzzle: I look for patterns in color or shapes or perhaps something that just doesn’t look right, an anomaly. My thought process can take those patterns and make sense of them, using heuristics I have developed to help me solve a puzzle.

——珍妮特

—Janet

就像先把外面的部分拼在一起来解决拼图游戏一样,我们可以使用基于会话的测试来为我们提供工作框架。在基于会话的测试中,我们创建一个任务或章程,然后为我们的会话设定时间限制,这样我们就可以专注于重要的事情。作为测试人员,我们经常会偏离轨道并最终追逐一个对我们当前正在测试的内容可能重要或不重要的错误。

Like solving the jigsaw puzzle by putting together the outside pieces first, we can use session-based testing to give us the framework in which we work. In session-based testing, we create a mission or a charter and then time-box our session so we can focus on what’s important. Too often as testers, we can go off track and end up chasing a bug that might or might not be important to what we are currently testing.

会话分为三种任务:测试设计和执行、错误调查和报告以及会话设置。我们衡量在设置上花费的时间与实际测试执行上花费的时间,以便我们知道在哪些方面花费的时间最多。我们可以以一致的方式捕获结果,以便我们可以向团队报告。

Sessions are divided into three kinds of tasks: test design and execution, bug investigation and reporting, and session setup. We measure the time we spend on setup versus actual test execution so that we know where we spend the most time. We can capture results in a consistent manner so that we can report back to the team.

有关基于会话的测试的更多信息,请查看 Jonathan Bach 的工作参考书目。

For more information on session-based testing, check the bibliography for work by Jonathan Bach.

自动化和探索性测试

Automation and Exploratory Testing

我们也可以将探索性测试与测试自动化结合起来。Jonathan Kohl 在他的文章“Man and Machine”[2007] 中谈到了交互式测试自动化以协助探索性测试。使用自动化来进行测试设置、数据生成、重复性任务,或者沿着工作流程前进到您想要开始的地方。然后你开始使用你的测试技能和经验来找到真正“好”的错误,那些以其他方式逃避注意的阴险的错误。您还可以使用自动化测试套件进行探索。稍微修改一下,观察运行结果,再次修改,观察会发生什么。

We can combine exploratory testing with test automation as well. Jonathan Kohl, in his article “Man and Machine” [2007], talks about interactive test automation to assist exploratory testing. Use automation to do test set up, data generation, repetitive tasks, or to progress along a workflow to the place you want to start. Then you start using your testing skills and experience to find the really “good” bugs, the insidious ones that otherwise escape attention. You can also use an automated test suite to explore. Just modify it a bit, watch the results as it runs, modify it again, and watch what happens.

探索性测试仪

An Exploratory Tester

通过探索性测试,每个测试人员对问题都有不同的方法,并且具有独特的工作风格。但是,有某些属性可以成为一个好的探索性测试人员。一个好的测试员:

With exploratory testing, each tester has a different approach to a problem, and has a unique style of working. However, there are certain attributes that make for a good exploratory tester. A good tester:

图像是系统的,但追求“气味”(异常,不一致的片段)

Is systematic, but pursues “smells” (anomalies, pieces that aren’t consistent)

图像通过使用 Oracles 学习识别问题(我们识别问题的原理或机制)

Learns to recognize problems through the use of Oracles (principle or mechanism by which we recognize a problem)

图像 选择一个主题或角色或使命宣言来集中测试

Chooses a theme or role or mission statement to focus testing

图像时间限制会议和边路旅行

Time-boxes sessions and side trips

图像考虑专家或新手用户会做什么

Thinks about what the expert or novice user would do

图像与领域专家一起探索

Explores together with domain experts

图像检查类似或有竞争力的应用程序

Checks out similar or competitive applications

探索性测试帮助我们了解应用程序的行为。测试人员通常对他们正在测试的应用程序了解很多。他们如何判断该应用程序是否可供技术水平较低或不熟悉的用户使用?可用性测试对于许多软件系统来说都是至关重要的。我们将在下一节中讨论。

Exploratory testing helps us learn about the behavior of an application. Testers generally know a lot about the application they’re testing. How do they judge whether the application is usable by users who are less technical or not familiar with it? Usability testing is vital for many software systems. We’ll talk about that in the next section.

可用性测试

Usability Testing

有两种类型的可用性测试。第一种类型是由用户体验人员预先完成的类型,使用线框等工具来帮助驱动编程。这些类型的测试属于第 2 象限。在本节中,我们将讨论批评产品的可用性测试类型。我们使用角色等工具和我们的直觉来帮助我们在考虑最终用户的情况下看待产品。

There are two types of usability testing. The first type is the kind that is done up front by the user experience folks, using tools such as wire frames to help drive programming. Those types of tests belong in Quadrant 2. In this section, we’re talking about the kind of usability testing that critiques the product. We use tools such as personas and our intuition to help us look at the product with the end user in mind.

用户需求和角色测试

User Needs and Persona Testing

让我们看一个网上购物的例子。我们考虑谁将使用该网站。是以前网上购物过的人,还是不知道如何进行的全新用户?我们猜测它将是两者的混合体,以及其他的。花时间询问您的营销团队以获取最终用户的人口统计数据。这些数字可能会帮助您计划测试。

Let’s look at an online shopping example. We think about who will use the site. Will it be people who have shopped online before, or will it be brand new users who have no idea how to proceed? We’re guessing it will be a mixture of both, as well as others. Take the time to ask your marketing group to get the demographics of the end users. The numbers might help you plan your testing.

使用角色的一种方法是让您的团队为您的应用程序发明几个不同的用户,代表不同的经验水平和需求。对于我们的 Internet 零售应用程序,我们可能具有以下角色:

One approach to using personas is for your team to invent several different users of your application representing different experience levels and needs. For our Internet retail application, we might have the following personas:

图像Nancy Newbie,一位刚接触网络购物并担心身份被盗用的老年人

Nancy Newbie, a senior citizen who is new to Internet shopping and nervous about identity theft

图像Hudson Hacker,寻找欺骗结帐页面的方法

Hudson Hacker, who looks for ways to cheat the checkout page

图像Enrico Executive,他在网上购物并向全球所有客户寄送礼物

Enrico Executive, who does all his shopping online and ships gifts to all his clients worldwide

图像Betty Bargain,正在寻找超值优惠

Betty Bargain, who’s looking for great deals

图像黛比·迪瑟勒 (Debbie Ditherer),她很难决定自己真正想要订购哪些商品

Debbie Ditherer, who has a hard time deciding what items she really wants to order

我们可能会在我们的工作区域挂上代表这些不同角色的照片和他们的传记,以便我们始终牢记它们。我们可以依次测试每个角色相同的场景,看看他们可能会遇到什么不同的体验。

We might hang photos representing these different personas and their biographies in our work area so that we always keep them in mind. We can test the same scenario as each persona in turn and see what different experiences they might encounter.

我们从 Brian Marick 和 Elisabeth Hendrickson 那里学到的另一种进行角色测试的方法是选择一个虚构人物或名人,并想象他们将如何使用我们的应用程序。英格兰女王能否驾驭我们的结帐流程?Homer Simpson 将如何搜索他想要的物品?

Another way to approach persona testing, which we learned from Brian Marick and Elisabeth Hendrickson, is to pick a fictional character or famous celebrity and imagine how they would use our application. Would the Queen of England be able to navigate our checkout process? How might Homer Simpson search for the item he wants?

在探索应用程序时,您也可以只担任新手、中级和专家用户的角色。用户能否在没有说明的情况下弄清楚他们应该做什么?如果您有很多新用户,您可能需要使界面非常简单。

You can also just assume the roles of novice, intermediate, and expert users as you explore the application. Can users figure out what they are supposed to do without instructions? If you have a lot of first-time users, you might need to make the interface very simple.

珍妮特的故事

Janet’s Story

当我第一次开始测试一个新的生产会计系统时,我发现流程很难理解,但团队中的生产会计很喜欢它。使用一段时间后,我了解了该应用程序背后的复杂性,并且知道为什么它不必对初次使用的用户来说是直观的。这对我来说是一个很好的教训,因为我一直认为应用程序必须是用户友好的。

When I first started testing a new production accounting system, I found it very difficult to understand the flow, but the production accountants on the team loved it. After I worked with it for a while, I understood the complexity behind the application and knew why it didn’t have to be intuitive for a first-time user. This was a good lesson for me, because I always assumed applications had to be user-friendly.

——珍妮特

—Janet

如果您的应用程序是为特定类型的用户定制的,它可能需要“智能”而不是直观。培训课程可能足以让克服最初缺乏可用性的问题,以便可以将界面设计为最大的效率和实用性。

If your application is custom-built for specific types of users, it might need to be “smart” rather than intuitive. Training sessions might be sufficient to get over the initial lack of usability so that the interface can be designed for maximum efficiency and utility.

导航

Navigation

导航是可用性测试的另一个方面。测试链接并确保 Tab 键顺序合理非常重要。如果用户可以选择应用程序或网站,并且初次体验不佳,他们可能不会再次使用您的应用程序。其中一些测试是可自动化的,但测试实际用户体验很重要。

Navigation is another aspect of usability testing. It’s incredibly important to test links and make sure the tabbing order makes sense. If a user has a choice of applications or websites, and has a bad first experience, they likely won’t use your application again. Some of this testing is automatable, but it’s important to test the actual user experience.

如果您有权访问最终用户,请让他们参与测试导航。与真实用户配对,或观看用户实际使用该应用程序并做笔记。在设计新的用户界面时,请考虑使用焦点小组来评估不同的界面。您可以从在纸上绘制的模型和流程开始,征求意见,然后尝试 HTML 模型,以获得早期反馈。

If you have access to the end users, get them involved in testing the navigation. Pair with a real user, or watch one actually use the application and take notes. When you’re designing a new user interface, consider using focus groups to evaluate different interfaces. You can start with mock-ups and flows drawn on paper, get opinions, and try HTML mock-ups next, to get early feedback.

请参阅第 8 章支持团队的面向业务的测试”,以了解绿野仙踪测试的示例,这是一种可用性设计方法。

See Chapter 8, “Business-Facing Tests that Support the Team,” for an example of Wizard of Oz testing, which is one approach to designing for usability.

查看比赛

Check Out the Competition

在评估您的应用程序的可用性时,请考虑其他类似的应用程序。他们如何完成任务?您认为它们是用户友好的还是直观的?如果您可以访问竞争软件,请花一些时间研究这些应用程序的工作原理并将它们与您的产品进行比较。例如,您正在测试一个采用日期范围的用户界面,并且它具有用于选择日期的弹出式日历功能。看看类似的日历功能如何在航空公司预订网站上工作。

When evaluating your application for usability, think about other applications that are similar. How do they accomplish tasks? Do you consider them user-friendly or intuitive? If you can get access to competing software, take some time to research how those applications work and compare them with your product. For example, you’re testing a user interface that takes a date range, and it has a pop-up calendar feature to select the date. Take a look at how a similar calendar function works on an airline reservation website.

可用性测试是一个相当专业的领域。如果您正在制作一个内部应用程序供一些接受过使用培训的用户使用,您可能不需要在可用性测试上投入太多。如果您正在为一家电话公司编写在线目录帮助,可用性可能是您的主要关注点,因此您需要尽可能多地了解它,或者请一位可用性专家。

Usability testing is a fairly specialized field. If you’re producing an internal application to be used by a few users who will be trained in its use, you probably don’t need to invest much in usability testing. If you’re writing the online directory assistance for a phone company, usability might be your main focus, so you need to learn as much as you can about it, or bring in a usability expert.

请参阅参考书目以获取 Jeff Patton、Gerard Meszaros 和其他人关于可用性测试的文章的链接。

See the bibliography for links to articles by Jeff Patton, Gerard Meszaros, and others on usability testing.

图形用户界面的背后

Behind the GUI

在题为“人与机器”[2007 年] 的演讲中,Jonathan Kohl 谈到了测试界面的替代方案。与其总是考虑通过用户界面进行测试,不如考虑以其他方式解决问题。考虑从您可以接近的各个角度测试整个系统。考虑使用模拟器或仿真器等工具。

In a presentation titled “Man and Machine” [2007], Jonathan Kohl talked about alternatives for testing interfaces. Instead of always thinking about testing through the user interface, consider attacking the problem in other ways. Think about testing the whole system from every angle that you can approach. Consider using tools like simulators or emulators.

第 9 章支持团队的面向业务的测试工具包”提供了有关促进这些测试的工具的更多详细信息。

Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” provides more detail about tools that facilitate these tests.

API测试

API Testing

第 8 章和9 章中,我们讨论了 GUI 背后的测试以驱动开发。在本节中,我们将展示您可以扩展 API 测试以尝试不同的排列和组合。

In Chapter 8 and Chapter 9, we talked about testing behind the GUI to drive development. In this section, we show that you can extend your tests for the API in order to try different permutations and combinations.

API (应用程序编程接口)是可以由其他软件应用程序或组件执行的功能的集合。最终用户通常永远不会意识到 API 的存在;她只是与顶部的界面进行交互。

An API (application programming interface) is a collection of functions that can be executed by other software applications or components. The end user is usually never aware that an API exists; she simply interacts with the interface on top.

每个 API 调用都有一个特定的函数,其中包含许多接受不同输入的参数。每个变体都会返回不同的结果。简单的测试是简单的输入。当参数一起工作以提供许多可能的变化时,会出现更复杂的测试模式。有时参数是可选的,因此了解可能性很重要。对于输入和预期结果,还应考虑边界条件。例如,对参数使用有效和无效字符串,改变内容,改变字符串输入的长度。

Each API call has a specific function with a number of parameters that accept different inputs. Each variation will return a different result. The easy tests are simple inputs. The more complicated testing patterns occur when the parameters work together to give many possible variations. Sometimes parameters are optional, so it’s important that you understand the possibilities. Boundary conditions should be considered as well, for both the inputs and expected results. For example, use both valid and invalid strings for parameters, vary the content, and vary the length of the strings’ input.

另一种测试方法是改变 API 调用的顺序。更改顺序可能会产生意想不到的结果,并揭示通过 UI 测试永远不会发现的错误。与使用 UI 相比,您可以更轻松地控制测试。

Another way to test is to vary the order of the API calls. Changing the sequence might produce unexpected results and reveal bugs that would never be found through UI testing. You can control the tests much more easily than when using the UI.

丽莎的故事

Lisa’s Story

我的团队正在研究一组故事,以使退休计划发起人能够上传工资单贡献文件。我们编写了 FitNesse 测试用例来说明文件解析规则,程序员也为此编写了单元测试。当解析器的编码完成后,我们想在解析器中输入更多的数据组合,包括一些非常奇怪的组合,然后看看会发生什么。我们可以使用我们用于测试的相同夹具来驱动开发,输入我们能想到的所有疯狂组合,然后查看结果。我们测试了大约 100 个有效和无效数据的变体。图 10-3显示了我们尝试过的几个测试的示例。通过这种方式,我们在代码中发现了几个错误。

My team was working on a set of stories to enable retirement plan sponsors to upload payroll contribution files. We wrote FitNesse test cases to illustrate the file parsing rules, and the programmer wrote unit tests for those as well. When the coding for the parser was complete, we wanted to throw a lot more combinations of data at the parser, including some really bizarre ones, and see what happened. We could use the same fixture as we used for our tests to drive development, enter all of the crazy combinations we could think of, and see the results. We tested about 100 variations of both valid and invalid data. Figure 10-3 shows an example of just a few of the tests we tried. We found several errors in the code this way.

图10-3解析规则测试示例

Figure 10-3 Sample of parsing rules test

图像

我们没有将所有这些测试都保留在回归套件中,因为它们只是一种快速尝试我们能想到的每种组合的方法。我们也可以以半自动、即席的方式完成这些测试,而不必费心将预期结果输入结果检查表,而只是盯着输出以确保它们看起来正确。

We didn’t keep all of these tests in the regression suite because they were just a means of quickly trying every combination we could think of. We could have done these tests in a semi-automated, ad hoc manner too, not bothering to type the expected results into the result checking table, and just eyeballing the outputs to make sure they looked correct.

——丽莎

—Lisa

珍妮特的故事

Janet’s Story

我最近使用了一个 Web 应用程序,该应用程序通过定义明确的 API 连接到遗留系统。由于遗留系统的设计和数据难以复制的事实,该团队尚未找到自动化此测试的方法。但是,我们可以查看日志文件以验证是否传递了正确的输入并返回了预期的结果。无论有没有自动化的好处,都可以对 API 进行有价值的探索性测试。

I recently worked with a web application that interfaces to a legacy system through a well-defined API. Due to the design of the legacy system and the fact that the data is hard to replicate, the team hasn’t yet found a way to automate this testing. However, we could look in the log files to verify the correct inputs were passed and the expected result was returned. Valuable exploratory testing of APIs is possible with or without benefit of automation.

——珍妮特

—Janet

API 调用可以在应用程序生命周期的早期开发,这意味着测试也可以在早期进行。通过 API 进行测试可以在开发 UI 之前增强对系统的信心。因为这种类型的测试可以自动化,所以您需要与您的程序员一起了解所有参数和每个函数的用途。如果您的程序员或自动化团队开发了易于使用的测试工具,您应该能够有条不紊地创建一套测试用例来执行该功能。

API calls can be developed early in an application life cycle, which means testing can occur early as well. Testing through an API can give confidence in the system before a UI is ever developed. Because this type of testing can be automated, you will need to work with your programmers to understand all of the parameters and the purpose of each function. If your programmers or automation team develop a test harness that is easy to use, you should be able to methodically create a suite of test cases that exercises the functionality.

网页服务

Web Services

Web 服务是一种基于服务的体系结构,它提供一个外部接口以便其他人可以访问系统。可能有多个利益相关者,您甚至可能不知道谁会使用您的产品。您的测试需要确认外部客户期望的服务质量。

Web services are a services-based architecture that provides an external interface so that others can access the system. There might be multiple stakeholders, and you may not even know who will be using your product. Your testing will need to confirm the quality of service that the external customers expect.

Web 服务通常需要大量的安全、压力和可靠性测试。有关这些类型的测试的更多信息,请参阅第 12 章“使用面向技术的测试来评价产品”。

Web services generally require plenty of security, stress, and reliability testing. See Chapter 12, “Critiquing the Product using Technology-Facing Tests,” for more on these types of tests.

在创建测试计划时考虑向客户承诺的服务级别。花时间进行探索性测试,以模拟用户可能访问 Web 服务的不同方式。

Consider levels of service that have been promised to clients when you are creating your test plans. Make time for exploratory testing to simulate the different ways users might access the web services.

Web 服务标准的使用还为当前的测试工具提供了其他含义。与 API 调用一样,基于 Web 服务的集成突出了验证接口点的重要性。但是,我们还需要考虑消息格式和处理、排队时间和消息响应时间。

The use of web services standards also offers other implications for current testing tools. As with API calls, web services-based integration highlights the importance of validating interface points. However, we also need to consider message formats and processing, queuing times, and message response times.

使用利用 GUI 驱动的自动化的测试工具对于 Web 服务项目来说是完全不够的。一种在“幕后”封装实现细节的特定领域语言非常适合测试 Web 服务。

Using testing tools that utilize GUI-driven automation is simply inadequate for a web services project. A domain-specific language that encapsulates implementation details “behind the scenes” works well for testing web services.

第 12 章,“测试象限总结”,有一个测试 Web 服务的例子。

Chapter 12, “Summary of Testing Quadrants,” has an example of testing web services.

测试文档和文档

Testing Documents and Documentation

在测试过程中经常被忽视的系统组件之一是文档。作为敏捷开发人员,我们可能更看重可工作的软件而不是文档,但我们仍然看重文档!用户手册和在线帮助与软件一样需要验证。您的团队可能会聘请专家,例如创建和验证文档的技术作家。与产品的所有其他组件一样,您的整个团队对文档的质量负责,包括硬拷贝和电子文档。

One of the components of the system that is often overlooked during testing is documentation. As agile developers, we may value working software over documentation, but we still value documentation! User manuals and online help need validation just as much as software. Your team may employ specialists such as technical writers who create and verify documentation. As with all other components of the product, your whole team is responsible for the quality of the documentation, and that includes both hard copy and electronic.

用户文档

User Documentation

您的团队可能会进行象限 2 测试以支持团队制作文档;事实上,我们鼓励这样做。Lisa 的团队编写的代码生成的文档的内容由政府法规指定,程序员可以先测试编写大部分代码。但是,自动化测试很难判断文档的格式是否正确或使用了可读的字体。他们也无法评估用户手册等文档的内容是否准确或有用。因为文档有很多主观成分,所以验证它更像是一种批评活动。

Your team might do Quadrant 2 tests to support the team as they produce documentation; in fact we encourage it. Lisa’s team writes code that produces documents whose contents are specified by government regulations, and programmers can write much of the code test-first. However, it’s difficult for automated tests to judge whether a document is formatted correctly or uses a readable font. They also can’t evaluate whether the contents of documents such as user manuals are accurate or useful. Because documentation has many subjective components, validating it is more of a critiquing activity.

珍妮特的故事

Janet’s Story

技术编写人员和测试人员可以紧密合作。Stephanie 是我在一个项目中合作过的一位技术作家,她与程序员交谈以了解应用程序的工作原理。她还将仔细检查申请表以确保她正确地写下了它。这似乎是测试工作的重复,所以斯蒂芬妮和我坐下来想出了一个更好的方法。

Technical writers and testers can work very closely together. Stephanie, a technical writer I worked with on one project, talked with the programmers to understand how the application worked. She would also work through the application to make sure she wrote it down correctly. This seemed to be a duplication of the testing effort, so Stephanie and I sat down and figured out a better approach.

我们决定在开发故事时共同努力。对于某些故事,斯蒂芬妮是首席“测试员”,有时我会担任该角色。如果我是领导,我会创建我的测试条件和示例,Stephanie 会使用这些作为文档的基础。当 Stephanie 担任领导时,她会编写文档,然后我会用它来确定测试用例。

We decided to work together on the stories as they were developed. For some stories Stephanie was lead “tester,” and sometimes I took that role. If I was lead, I’d create my test conditions and examples and Stephanie would use those as her basis for the documentation. When Stephanie was lead, she would write her documentation, and then I would use that to determine the test cases.

这样做可以使文档在执行之前得到测试,并使测试受到挑战。像这样携手合作被证明是一个非常成功的实验。生成的文档与软件的行为相匹配,对最终用户更有用。

Doing it this way enabled the documentation to be tested and the tests to be challenged before they were ever executed. Working hand in hand like this proved to be a very successful experiment. The resulting documentation matched the software’s behavior and was much more useful to the end users.

——珍妮特

—Janet

也不要忘记查看帮助文本。帮助文本的链接是否易于识别?它们在整个用户界面中是否一致?帮助文本是否清晰呈现?如果它在弹出窗口中打开,并且用户在他们的浏览器中阻止弹出窗口,会有什么影响?帮助是否涵盖了所有需要的主题?在 Lisa 的项目中,帮助文本的优先级往往较低,因此通常根本无法完成。这是一项业务决策,但如果您觉得应用程序的某个区域需要额外的帮助文本或文档,请向您的团队和客户提出问题。

Don’t forget to check the help text too. Are the links to help text easily identifiable? Are they consistent throughout the user interface? Is the help text presented clearly? If it opens in a pop-up, and users block pop-ups in their browsers, what’s the impact? Does the help cover all of the topics needed? On Lisa’s projects, help text tends to be a low priority, so it often doesn’t get done at all. That’s a business decision, but if you feel an area of the application needs extra help text or documentation, raise the issue to your team and your customers.

报告

Reports

从测试的角度来看,另一个经常被忽视的系统组件是报告。出于决策目的,报告对许多用户来说至关重要,但通常留到最后,要么没有完成,要么执行不力。可以定制报告以满足特定的客户需求,但是有许多第三方工具可用于生成报告。报告可以是应用程序本身的一部分,也可以通过单独的报告系统为最终用户生成。

Another system component that’s often overlooked from a testing perspective is reports. Reports are critical to many users for decision-making purposes but are often left until the very end, and either don’t get done or are poorly executed. Reports might be tailored to meet specific customer needs, but there are many third-party tools available for generating reports. Reports may be part of the application itself or be generated through a separate reporting system for end users.

我们与其他象限 3 测试活动一起讨论测试报告,以评价产品,但我们建议您也编写象限 2 报告测试,以指导编码并帮助团队在生成报告时了解客户的需求。它们当然可以写成测试先行的。但是,与文档一样,您需要查看报告以了解它是否足够容易阅读并以易于理解的方式呈现信息。

We discuss testing reports along with the other Quadrant 3 test activities in order to critique the product, but we recommend that you also write Quadrant 2 report tests that will guide the coding and help the team understand the customer’s needs as it produces reports. They can certainly be written test-first. Like documents, though, you need to look at a report to know if it’s easy enough to read and presents information in an understandable way.

测试报告时最大的挑战之一不是格式化,而是获取正确的数据。当您尝试为报告创建测试数据时,可能很难获得真实数据的良好横截面。导致报告失败的通常也是边缘情况,因此合并这些额外数据是不可行的。在大多数情况下,最好使用生产数据(或从生产系统复制到测试环境的数据)来测试不同的报告变体。

One of the biggest challenges when testing reports is not the formatting but getting the right data. When you try to create test data for reports, it can be difficult to get a good cross section of realistic data. It also is usually the edge cases that make the reports fail, so incorporating that extra data is not feasible. In most cases, it’s best to use production data (or data copied from the production system into a test environment) to test the different reporting variations.

丽莎的故事

Lisa’s Story

我们的应用程序包括许多报告,其中许多帮助公司满足政府合规要求。虽然我们对每个报告都有自动冒烟测试,但对报告的任何更改,甚至是我们用于生成报告的工具的升级,都需要进行大量的手动和可视化测试。我们必须像鹰一样观察:一个数字是否被一个字符截断了?有一段文字跑到下一页了吗?是否包含正确的数据?错误或缺失的数据可能意味着监管机构有麻烦。

Our application includes a number of reports, many of which help companies meet governmental compliance requirements. While we have automated smoke tests for each report, any change to a report, or even an upgrade in the tool we use to generate reports, requires extensive manual and visual testing. We have to watch like hawks: Has a number been truncated by one character? Did a piece of text run over to the next page? Is the right data included? Wrong or missing data can mean trouble with the regulatory agency.

另一个挑战是验证报告中包含的数据。如果我要使用报告使用的相同查询,它并不能证明任何事情。有时我很难想出自己的 SQL 查询来将实际数据与报告中显示的数据进行比较。我们会安排额外的时间来测试报告,即使是看起来很简单的报告。

Another challenge is verifying the data contained in the report. If I were to use the same query that the report uses, it doesn’t prove anything. I sometimes struggle to come up with my own SQL queries to compare the actual data with what shows up on a report. We budget extra time to test reports, even the simple-looking ones.

由于报告非常主观,我们发现不同的利益相关者对数据的呈现方式有不同的偏好。必须通过电话向用户解释报告的计划管理员与决定报告中需要包含哪些数据的公司律师相比,对于什么容易理解有不同的想法。我们的产品负责人帮助我们从业务的各个领域获得共识。

Because reports are so subjective, we find that different stakeholders have different preferences for how the data is presented. The plan administrator who has to explain a report to a user on the phone has a different idea of what’s easy to understand than the company lawyer who decides what data needs to be on the report. Our product owner helps us get consensus from all areas of the business.

报告的内容和格式当然很重要,但对于在线报告来说,它们出现的速度也很关键。我们的计划管理员希望完全自由地为某些交易历史报告指定任何日期范围。我们编写报告的 DBA 警告说,对于一家大公司的退休计划,价值超过几个月的交易数据可能需要几分钟才能呈现。随着时间的推移,公司不断壮大,交易越来越多,最终用户界面在提供报告之前开始超时。测试时,尝试最坏的情况,这最终可能成为最常见的情况。

The contents and formatting of a report are important, of course, but for online reports, the speed at which they come up is critical too. Our plan administrators wanted complete freedom to specify any date range for some transaction history reports. Our DBA, who coded the reports, warned that for a large company’s retirement plan, data for more than a few months worth of transactions could take several minutes to render. Over time, companies grew, they had more and more transactions, and eventually the user interface started timing out before it could deliver the report. When testing, try out worst-case scenarios, which could eventually become the most common scenario.

——丽莎

—Lisa

如果您正在处理一个涉及大量报告的项目,请不要屈服于将它们留到最后的诱惑。如果可以的话,在每次迭代中包含一些报告。一份报告可以是一个故事,也可以分解成几个故事。使用模型帮助客户决定报告内容和格式。在报告中找到“薄切片”或“关键路径”,首先对其进行编码,并在添加下一个切片之前将其展示给您的客户。增量开发与报表一样适用于其他软件。

If you’re tackling a project that involves lots of reports, don’t give in to the temptation to leave them to the end. Include some reports in each iteration if you can. One report could be a single story or maybe even broken up into a couple of stories. Use mock-ups to help the customers decide on report contents and formatting. Find the “thin slice” or “critical path” in the report, code that first, and show it to your customer before you add the next slice. Incremental development works as well with reports as it does with other software.

有关使用薄切片的信息,请参阅第 8 章支持团队的面向业务的测试”。

See Chapter 8, “Business-Facing Tests that Support the Team,” for information about using thin slices.

有时,您的客户自己并不确定报告的外观或如何逐步处理报告。有时团队中没有人预料到测试工作会有多困难。

Sometimes your customers themselves aren’t sure how a report should look or how to approach it incrementally. And sometimes nobody on the team anticipates how hard the testing effort will prove to be.

丽莎的故事

Lisa’s Story

与其他金融账户一样,退休计划需要向账户持有人提供定期报表,详细说明所有进出账户的资金。这些报表显示了期初和期末余额之间的价值变化以及其他相关信息,例如账户受益人的姓名。我们公司希望改进账户报表,既可以作为一种营销工具,也可以减少不理解报表的账户持有人打来的电话。

Like other financial accounts, retirement plans need to provide periodic statements to account holders that detail all of the money going into and out of the account. These statements show the change in value between the beginning and ending balances and other pertinent information, such as the names of account beneficiaries. Our company wanted to improve the account statements, both as a marketing tool and to reduce the number of calls from account holders who didn’t understand their statements.

我们无法访问直接竞争对手的账户报表,因此产品所有者要求志愿者从银行和其他金融机构获取账户报表以获取想法。数月的讨论和模型实验产生了一种新的报表格式,其中包括以前报告中没有的数据,例如每个共同基金的业绩结果。

We didn’t have access to our direct competitors’ account statements, so the product owner asked for volunteers to bring in account statements from banks and other financial institutions in order to get ideas. Months of discussions and experimentation with mock-ups produced a new statement format, which included data that wasn’t on the report previously, such as performance results for each mutual fund.

开发新账户报表的故事分布在两个季度的迭代中。在第一季度,完成了收集新数据的故事。测试证明比我们想象的要难得多。我们使用 FitNesse 测试来验证捕获不同的数据元素,这让我们产生了一种错误的安全感。很难覆盖所有的变化,我们在自动化测试中遗漏了一些。我们也没有预料到收集新数据的变化会对现有报表上已经显示的数据产生不利影响。

Stories for developing the new account statement were distributed throughout two quarters worth of iterations. During the first quarter, stories to collect new data were done. Testing proved much harder than we thought. We used FitNesse tests to verify capturing the different data elements, which lulled us into a false sense of security. It was hard to cover all of the variations, and we missed some with the automated tests. We also didn’t anticipate that the changes to collect new data could have an adverse effect on the data that already displayed on the existing statements.

因此,我们没有对账户报表进行充分的手动测试。细微的错误从我们身边溜走。当制作季度报表的工作开始时,客户开始打来电话。我们疯狂地争先恐后地诊断和修复代码和数据中的错误。整个项目被推迟了四分之一,而我们想出了更好的测试方法并添加了内部检查和更好的代码日志记录。

As a result, we didn’t do adequate manual testing of the account statements. Subtle errors slipped past us. When the job to produce quarterly statements ran, calls started coming in from customers. We had a mad scramble to diagnose and fix the errors in both code and data. The whole project was delayed by a quarter while we figured out better ways to test and added internal checks and better logging to the code.

——丽莎

—Lisa

短迭代意味着很难腾出时间进行充分的探索性测试和其他象限 3 活动。让我们看看可能有助于加快此测试并为重要的手动和可视化测试腾出时间的工具。

Short iterations mean that it can be hard to make time for adequate exploratory testing and other Quadrant 3 activities. Let’s look at tools that might help speed up this testing and make time for vital manual and visual tests.

辅助探索性测试的工具

Tools to Assist with Exploratory Testing

探索性测试是手动测试。一些最好的测试之所以发生,是因为一个人正在关注细节,而如果我们关注这些细节,这些细节往往会被遗漏一个脚本。直觉是我们无法让机器学习的东西。然而,有许多工具可以帮助我们追求卓越。

Exploratory testing is manual testing. Some of the best testing happens because a person is paying attention to details that often get missed if we are following a script. Intuition is something that we cannot make a machine learn. However, there are many tools that can assist us in our quest for excellence.

工具不应该取代人际互动;他们应该增强体验。工具可以为测试人员提供更多的能力来发现难以重现的错误,这些错误通常会被归档,因为没有人能够处理它们。探索性测试是非常规的,那么为什么工具不能如此呢?考虑可以将工具整合到您的测试中的省力、高价值的方式。

Tools shouldn’t replace human interaction; they should enhance the experience. Tools can provide testers with more power to find the hard-to-reproduce bugs that often get filed away because no one can get a handle on them. Exploratory testing is unconventional, so why shouldn’t the tools be as well? Think about low-effort, high-value ways that tools can be incorporated into your testing.

计算机擅长执行重复性任务和执行计算。这是它们比人类强得多的两个领域,所以让我们用它们来完成这些任务。因为测试需要跟上编码的步伐,所以我们可以获得的任何时间优势都是一种奖励。

Computers are good at doing repetitive tasks and performing calculations. These are two areas where they are much better than humans, so let’s use them for those tasks. Because testing needs to keep pace with coding, any time advantage we can gain is a bonus.

请参阅参考书目,了解 Jonathan Kohl 关于结合使用人力和自动化力量进行最佳测试的著作。

See the bibliography for references to Jonathan Kohl’s writings on using human and automation power together for optimal testing.

在接下来的几节中,我们将研究自动化可以利用探索性测试的一些领域。我们涵盖的内容是测试设置、测试数据生成、监控、模拟器和模拟器。

In the next few sections, we’ll look at some areas where automation can leverage exploratory testing. The ones we cover are test setup, test data generation, monitoring, simulators, and emulators.

测试设置

Test Setup

让我们想想我们在测试时做了什么。我们刚刚发现了一个错误,但不是一个容易重现的错误。我们非常确定它是组件之间交互的结果。我们回到起点,尝试一个又一个场景。很快我们就花了一整天的时间来重现这个错误。

Let’s think about what we do when we test. We’ve just found a bug, but not one that is easily reproducible. We’re pretty sure it happens as a result of interactions between components. We go back to the beginning and try one scenario after another. Soon we’ve spent the whole day just trying to reproduce this one bug.

问问自己如何使这更容易。我们发现最耗时的任务之一是测试设置和为实际测试找到正确的起点。如果您使用基于会话的测试,那么您已经知道您花了多少时间来设置测试,因为您一直在跟踪那个特定的时间浪费。这是一些自动化的绝佳机会。

Ask yourself how you can make this easier. We’ve found that one of the most time-consuming tasks is the test setup and getting to the right starting point for your actual test. If you use session-based testing, then you already know how much time you spend setting up the test, because you have been tracking that particular time waster. This is an excellent opportunity for some automation.

用于支持第 9 章中描述的团队的面向业务的测试的工具对于手动探索性测试也很有价值。可以运行自动化功能测试脚本来设置数据和场景以启动探索性测试会话。配置为接受运行时参数的测试对于设置评估产品的起点特别有用。

The tools used for business-facing tests that support the team described in Chapter 9 are also valuable for manual exploratory testing. Automated functional test scripts can be run to set up data and scenarios to launch exploratory testing sessions. Tests configured to accept runtime parameters are particularly powerful for setting up a starting point for evaluating the product.

丽莎的故事

Lisa’s Story

我们的 Watir 测试脚本都接受许多运行时参数。当我需要一个包含一组特定选项和特定类型参与者的退休计划时,我可以启动一个或两个 Watir 脚本,并在命令行上设置一些变量。当脚本停止时,我有一个浏览器会话,其中已经设置了测试所需的所有数据。这是如此之快,以至于我可以测试我永远无法使用全手动击键的排列。

Our Watir test scripts all accept a number of runtime parameters. When I need a retirement plan with a specific set of options, and specific types of participants, I can kick off a Watir script or two with some variables set on the command line. When the scripts stop, I have a browser session with all of the data I need for testing already set up. This is so fast that I can test permutations I’d never get to using all-manual keystrokes.

——丽莎

—Lisa

您用于功能回归测试和指导开发的测试脚本并不是唯一可以帮助您摆脱单调乏味的手动探索性测试的工具。还有其他工具可帮助设置测试数据以及帮助您评估测试会话的输出。

The test scripts you use for functional regression testing and for guiding development aren’t the only tools that help take the tedium out of manual exploratory testing. There are other tools to help set up test data as well as to help you evaluate the outputs of your testing sessions.

无论您使用的是什么工具,请考虑如何调整它以在插入不同输入的情况下一遍又一遍地运行场景。Janet 还成功地使用 Ruby 和 Watir 来设置测试以运行多次以帮助识别错误。以与最终用户大致相同的方式驱动浏览器或 UI 的工具将使您的测试更加可靠,因为您可以在显示器上回放它并观察在设置过程中可能看起来不正常的任何东西。当你到达测试真正开始的地方时,你就可以利用你出色的测试能力来追踪错误的来源。

Whatever tool you are using, think about how it can be adapted to run the scenario over and over with different inputs plugged in. Janet has also successfully used Ruby with Watir to set up tests to run multiple times to help identify bugs. Tools that drive the browser or UI in much the same way that an end user would makes your testing more reliable because you can play it back on your monitor and watch for anything that might not look as it should during the setup. When you get to the place where the test actually starts, you can then use your excellent testing abilities to track down the source of the bug.

测试数据生成

Test Data Generation

PerlClip 是一个工具示例,您可以使用它来测试具有不同类型输入的文本字段。James Bach 在他的网站www.satisfice.com上免费提供它,它对验证字段非常有帮助。例如,如果您有一个最多接受 200 个字符输入的字段,手动测试该字段及其边界将非常繁琐。使用 PerlClip 创建一个字符串,将其放入您的自动化库中,然后让您的自动化工具调用该字符串来测试该值。

PerlClip is an example of a tool that you can use to test a text field with different kinds of inputs. James Bach provides it free of charge on his website, www.satisfice.com, and it can be very helpful in validating fields. For example, if you have a field that will accept a maximum input of 200 characters, testing this field and its boundaries manually would be very tedious. Use PerlClip to create a string, put it in your automation library, and have your automation tool call the string to test the value.

监控工具

Monitoring Tools

Unix/Linux 命令tail -f或 James Bach 的 LogWatch 等工具可以帮助监视日志文件中的错误情况。IDE 还提供日志分析工具。许多错误消息永远不会显示在屏幕上,因此如果您通过 GUI 进行测试,您永远不会看到它们。熟悉这些工具,因为它们可以使您的测试更加有效和高效。如果你不是要确定您的系统在哪里记录警告和错误,请询问您的开发人员。他们可能对如何监控系统有很多想法。

Tools like the Unix/Linux command tail -f, or James Bach’s LogWatch, can help monitor log files for error conditions. IDEs also provide log analysis tools. Many error messages are never displayed on the screen, so if you’re testing via the GUI, you never see them. Get familiar with tools like these, because they can make your testing more effective and efficient. If you are not sure where your system logs warnings and errors, ask your developers. They probably have lots of ideas about how you can monitor the system.

模拟器

Simulators

模拟器是用于创建数据的工具,这些数据代表被测系统的真实数据的关键特征和行为。如果您无法访问系统的真实数据,模拟数据有时也几乎可以正常工作。使用模拟器的另一个优势是随着时间的推移将数据泵入系统。它可用于帮助生成在正常情况下难以创建的错误条件,并可减少边界测试的时间。

Simulators are tools used to create data that represent key characteristics and behavior of real data for the system under test. If you do not have access to real data for your system, simulated data will sometimes work almost as well. The other advantage of using a simulator is for pumping data into a system over time. It can be used to help generate error conditions that are difficult to create under normal circumstances and can reduce time in boundary testing.

请参阅第 12 章“测试象限总结”中的“系统测试”示例,了解模拟器对于测试整个系统的重要性。

See the “System Test” example in Chapter 12, “Summary of Testing Quadrants,” to see how a simulator was critical to the testing the whole system.

设置数据和测试场景只是成功的一半。您还需要有一种方法来观察您的测试结果。让我们考虑一些用于此目的的工具。

Setting up data and test scenarios is half of the picture. You also need to have a way to watch the outcomes of your testing. Let’s consider some tools for that purpose.

模拟器

Emulators

仿真器复制系统的功能,使其表现得像被测系统。使用模拟器的原因有很多。当您需要测试与其他系统或设备接口的代码时,仿真器是非常宝贵的。

An emulator duplicates the functionality of a system so that it behaves like the system under test. There are many reasons to use an emulator. When you need to test code that interfaces with other systems or devices, emulators are invaluable.

仿真器是一种有助于保持测试和编码同步进行的工具。使用它们是测试以跟上短迭代开发的一种方法。在计划发布和迭代时,请考虑可能有助于创建类似生产的测试场景的工具类型。看看您是否可以使用您已经用于自动化测试的工具来推动开发,作为探索性测试的辅助工具。

Emulators are one tool that helps to keep testing and coding moving together hand-in-hand. Using them is one way for testing to keep up with development in short iterations. As you plan your releases and iterations, think about the types of tools that might help with creating production-like test scenarios. See if you can use the tools you’re already using for automating tests to drive development as aids to exploratory testing.

通过测试推动开发对于任何项目的成功都是至关重要的。然而,我们人类不会总是完全正确地获得所需系统行为的所有要求。当我们的业务专家提供功能应该如何工作的示例时,他们自己可能会错过功能的重要方面或与系统其他部分的交互。我们必须使用技术来帮助客户和开发团队更多地了解系统,以便他们能够不断改进产品。

Driving development with tests is critical to any project’s success. However, we humans won’t always get all of the requirements for desired system behavior entirely correct. Our business experts themselves can miss important aspects of functionality or interaction with other parts of the system when they provide examples of how a feature should work. We have to use techniques to help both the customer and developer teams learn more about the system so they can keep improving the product.

概括

Summary

测试工作的很大一部分用于从业务角度对产品进行评价。本章为您提供了一些关于可以执行的测试类型的想法,以使您的测试工作更有效。

A large part of the testing effort is spent critiquing the product from a business perspective. This chapter gave you some ideas about the types of tests you can do to make your testing efforts more effective.

图像向利益相关者展示软件以获得早期反馈,这将有助于指导构建正确的东西。

Demonstrate software to stakeholders in order to get early feedback that will help direct building the right stuff.

图像使用场景和工作流从头到尾测试整个系统。

Use scenarios and workflows to test the whole system from end to end.

图像使用探索性测试来补充自动化并利用人类的智慧和感知。

Use exploratory testing to supplement automation and to take advantage of human intellect and perceptions.

图像如果在测试和编码时不考虑可用性,应用程序可能会变成货架软件。始终注意系统的使用方式。

Without usability in mind when testing and coding, applications can become shelfware. Always be aware of how the system is being used.

图像在 GUI 后面进行测试是获得应用程序功能的最有效方法。做一些研究以了解如何处理您的应用程序。

Testing behind the GUI is the most effective way of getting at the application functionality. Do some research to see how you can approach your application.

图像结合各种测试来制作一个好的回归套件。

Incorporate all kinds of tests to make a good regression suite.

图像不要忘记测试文档和报告。

Don’t forget about testing documentation and reports.

图像自动化工具可以执行繁琐和重复的任务,例如数据和测试场景设置,并腾出更多时间用于重要的手动探索性测试。

Automation tools can perform tedious and repetitive tasks, such as data and test scenario setup, and free up more time for important manual exploratory testing.

图像您已经用于自动化功能测试的工具也可能有助于利用探索性测试。

Tools you’re already using to automate functional tests might also be useful to leverage exploratory tests.

图像 操作系统和 IDE 中内置的监视、资源使用和日志分析工具可帮助测试人员评估应用程序的行为。

Monitoring, resource usage, and log analysis tools built into operating systems and IDEs help testers appraise the application’s behavior.

图像即使您无法复制确切的生产环境,模拟器和仿真器也可以进行探索性测试。

Simulators and emulators enable exploratory testing even when you can’t duplicate the exact production environment.

图像即使使用测试来推动开发,也可能会遗漏或误解对所需行为或与其他系统交互的要求。象限 3 的活动帮助团队不断为产品增加价值。

Even when tests are used to drive development, requirements for desired behavior or interaction with other systems can be missed or misunderstood. Quadrant 3 activities help teams keep adding value to the product.

第 11 章使用面向技术的测试来评价产品

Chapter 11 Critiquing the Product Using Technology-Facing Tests

图像

本章的重点是我们测试象限的右下角。我们研究了通过面向业务和面向技术的测试来推动开发。编写代码后,我们不再推动开发,而是寻找评价产品的方法。在上一章中,我们研究了从商业角度进行批评的方法。现在我们看看如何从面向技术的角度进行批评。这些测试是评估我们的产品是否提供正确商业价值的重要手段。

This chapter is focused on the bottom right corner of our testing quadrant. We’ve looked at driving development with both business-facing and technology-facing tests. After the code is written, we are no longer driving the development but are looking at ways to critique the product. In the previous chapter, we examined ways to critique from a business point of view. Now we look at ways to critique from a technology-facing point of view. These tests are an important means of evaluating whether our product delivers the right business value.

象限 4 简介

Introduction to Quadrant 4

单个故事是拼图的一部分,但应用程序的意义远不止于此。批评产品的面向技术的测试更多与功能需求相比,更关注非功能需求。我们从技术角度担心产品存在缺陷。我们没有使用业务领域语言,而是使用编程领域词汇表来描述需求。这是象限 4 的省份(见图11-1)。

Individual stories are pieces of the puzzle, but there’s more to an application than that. The technology-facing tests that critique the product are more concerned with the nonfunctional requirements than the functional ones. We worry about deficiencies in the product from a technical point of view. Rather than using the business domain language, we describe requirements using a programming domain vocabulary. This is the province of Quadrant 4 (see Figure 11-1).

图 11-1象限 4 测试

Figure 11-1 Quadrant 4 tests

图像

非功能性需求包括配置问题、安全性、性能、内存管理、“能力”(例如,可靠性、互操作性和可扩展性)、恢复,甚至数据转换。并非所有项目都关心所有这些问题,但最好有一个清单以确保团队考虑这些问题并询问客户每个问题的重要性。

Nonfunctional requirements include configuration issues, security, performance, memory management, the “ilities” (e.g., reliability, interoperability, and scalability), recovery, and even data conversion. Not all projects are concerned about all of these issues, but it is a good idea to have a checklist to make sure the team thinks about them and asks the customer how important each one is.

我们的客户应该考虑所有重要的质量属性和因素,并做出明智的权衡。然而,许多客户关注在应用程序的业务方面,不理解许多非功能性需求在帮助定义产品所需质量水平方面的重要性。他们可能会假设开发团队只会处理性能、可靠性和安全性等问题。

Our customer should think about all of the quality attributes and factors that are important and make informed trade-offs. However, many customers focus on the business side of the application and don’t understand the criticality of many nonfunctional requirements in their role of helping to define the level of quality needed for the product. They might assume that the development team will just take care of issues such as performance, reliability, and security.

我们认为开发团队有责任解释不解决这些非功能性或跨功能需求的后果。我们实际上都是想要提供良好价值的产品团队的一员,而这些以技术为导向的因素可能会暴露成败问题。

We believe that the development team has a responsibility to explain the consequences of not addressing these nonfunctional or cross-functional requirements. We’re really all part of one product team that wants to deliver good value, and these technology-oriented factors might expose make-or-break issues.

许多这些非功能性和跨功能性问题对于许多应用程序来说被认为是低风险的,因此不会添加到测试计划中。但是,当您计划项目时,您应该考虑每个领域的风险,在您的测试计划中解决它们,并将测试它们所需的工具和资源包括在您的项目计划中。

Many of these nonfunctional and cross-functional issues are deemed low-risk for many applications and so are not added to the test plan. However, when you are planning your project, you should think about the risks in each of these areas, address them in your test plan, and include the tools and resources needed for testing them in your project plan.

丽莎的故事

Lisa’s Story

过去,性能和安全测试等领域的专家曾问我,为什么他们在敏捷会议或有关敏捷开发的出版物中很少听到“能力”测试。和珍妮特一样,我一直认为这些测试领域很重要,所以这不是我的看法。但当我想到它时,我不得不承认这在当时并不是一个被广泛讨论的话题(尽管最近有所改变)。

In the past, I’ve been asked by specialists in areas such as performance and security testing why they didn’t hear much about “ility” testing at agile conferences or in publications about agile development. Like Janet, I’ve always seen these areas of testing as critical, so this wasn’t my perception. But as I thought about it, I had to agree that this wasn’t a much-discussed topic at the time (although that’s changed recently).

为什么敏捷讨论不包括负载测试这样重要的考虑因素?我的理论是,这是因为敏捷开发是由客户驱动的,来自用户故事。客户只是假设软件将被设计为以合理的性能速率适当地适应潜在的负载。他们并不总是想到用语言表达这些担忧。如果不要求解决它们,程序员可能会或可能不会考虑优先考虑它们。我相信测试人员对敏捷团队做出巨大贡献的一个领域是提出诸如“应用程序应支持多少并发用户?”之类的问题。和“所需的平均响应时间是多少?”

Why would agile discussions not include such important considerations as load testing? My theory is that it’s because agile development is driven by customers, from user stories. Customers simply assume that software will be designed to properly accommodate the potential load, at a reasonable rate of performance. It doesn’t always occur to them to verbalize those concerns. If not asked to address them, programmers may or may not think to prioritize them. I believe that one area where testers have contributed greatly to agile teams is in bringing up questions such as, “How many concurrent users should the application support?” and “What’s the average response time required?”

——丽莎

—Lisa

由于此象限中的测试类型多种多样,因此我们将提供可能对我们有用的工具示例,而不是单独的工具包部分。工具,无论是自产的还是收购的,都是第 4 象限测试工作取得成功所必需的。尽管如此,做工作的人还是很重要的,所以让我们考虑一下敏捷团队中的哪些人可以执行这些测试。

Because the types of testing in this quadrant are so diverse, we’ll give examples of tools that might be helpful as we go along instead of a separate toolkit section. Tools, whether homegrown or acquired, are essential to succeed with Quadrant 4 testing efforts. Still, the people doing the work count, so let’s consider who on an agile team can perform these tests.

谁做的?

Who Does It?

所有关于敏捷的文献都在谈论团队是多面手;任何人都应该能够选择一项任务并完成它。我们知道这并不总是实用的,但我们的想法是能够分享知识,这样人们就不会成为信息孤岛。

All of the agile literature talks about teams being generalists; anyone should be able to pick up a task and do it. We know that isn’t always practical, but the idea is to be able to share the knowledge so that people don’t become silos of information.

但是,有许多任务需要专业知识。一个很好的例子是安全测试。我们不是在谈论应用程序内的安全性,例如谁有权管理它。因为这种类型的安全性实际上是功能需求的一部分,并且会被常规故事所涵盖,所以验证它是否有效属于前三个象限。我们正在谈论探测外部安全漏洞并了解黑客利用的系统中的漏洞类型。那是一套专门的技能。

However, there are many tasks that need specialized knowledge. A good example is security testing. We’re not talking about security within an application, such as who has access rights to administer it. Because that type of security is really part of the functional requirements and will be covered by regular stories, verifying that it works falls within the first three quadrants. We’re talking about probing for external security flaws and knowing the types of vulnerabilities in systems that hackers exploit. That is a specialized skill set.

性能测试可以由测试人员和程序员合作完成,并为他们的特定需求构建简单的工具。一些组织购买负载测试工具,需要专门研究该工具的团队成员来构建脚本并分析和解释结果。软件开发组织(尤其是小型软件开发组织)可能很难拥有足够的资源来为测试复制准确的生产级负载,因此可能需要外部性能测试提供商。

Performance testing can be done by testers and programmers collaborating and building simple tools for their specific needs. Some organizations purchase load-testing tools that require team members who specialize in that tool to build the scripts and analyze and interpret the results. It can be difficult for a software development organization, especially a small one, to have enough resources to duplicate an accurate production-level load for a test, so external providers of performance testing may be needed.

较大的组织可能有一些小组,例如您的团队可以用来帮助进行数据转换的数据库专家,可以帮助您识别应用程序风险的安全小组,或者可以帮助您测试恢复或故障转移的生产支持团队。与这些专家建立密切的关系。你需要作为一个虚拟团队一起工作,以收集你需要的关于你的产品的信息。

Larger organizations may have groups such as database experts that your team can use to help with data conversion, security groups that will help you identify risks to your application, or a production support team that can help you test recovery or failover. Build a close relationship with these specialists. You’ll need to work together as a virtual team to gather the information you need about your product.

第 15 章,“发布或主题规划中的测试人员活动”,解释了如何规划与外部团队的合作。

Chapter 15, “Tester Activities in Release or Theme Planning,” explains how to plan to work with external teams.

您团队中的技能组合越多样化,您就越不可能需要外部顾问来帮助您。确定每个项目所需的资源。许多团队发现优秀的技术测试员或工具匠可以承担其中的许多任务。如果团队中已有人可以学习所需的任何专业知识,那就太好了;否则,请引入您需要的专业知识。

The more diverse the skill sets are in your team, the less likely you are to need outside consultants to help you. Identify the resources you need for each project. Many teams find that a good technical tester or toolsmith can take on many of these tasks. If someone already on the team can learn whatever specialized knowledge is required, great; otherwise, bring in the expertise you need.

无论您的团队是否为这些类型的测试引入额外资源,您的团队仍然有责任确保完成最少的测试。这些测试提供的信息可能会在某些领域产生新的故事和任务,例如更改架构以获得更好的可扩展性或实施系统范围的安全解决方案。确保完成从批评产品的测试到推动将改进产品非功能方面的更改的测试的反馈循环。

Regardless of whether or not your team brings in additional resources for these types of tests, your team is still responsible for making sure the minimum testing is done. The information these tests provide may result in new stories and tasks in areas such as changing the architecture for better scalability or implementing a system-wide security solution. Be sure to complete the feedback loop from tests that critique the product to tests that drive changes that will improve the nonfunctional aspects of the product.

仅仅因为这是四个敏捷测试象限中的第四个并不意味着这些测试排在最后。您的团队需要考虑何时进行性能、安全和“功能”测试,以确保您的产品提供正确的商业价值。

Just because this is the fourth out of four agile testing quadrants doesn’t mean these tests come last. Your team needs to think about when to do performance, security, and “ility” tests so that you ensure your product delivers the right business value.

你什么时候做?

When Do You Do It?

与功能测试一样,支持团队的面向技术的测试越早完成,修复发现的任何问题的成本就越低。然而,许多跨功能测试的成本很高,而且很难在小块中进行。

As with functional testing, the sooner technology-facing tests that support the team are completed, the cheaper it is to fix any issues that are found. However, many of the cross-functional tests are expensive and hard to do in small chunks.

可以编写技术故事来解决特定需求,例如:“作为用户 Abby,我需要在 20 秒内检索报告 X,以便我可以快速做出决定。” 这个故事是关于性能的,需要编写专门的测试,它可以与故事一起完成以对报告进行编码,或者在以后的迭代中完成。

Technical stories can be written to address specific requirements, such as: “As user Abby, I need to retrieve report X in less than 20 seconds so that I can make a decision quickly.” This story is about performance and requires specialized tests to be written, and it can be done along with the story to code the report, or in a later iteration.

考虑在你的故事板上单独一行来处理整个产品所需的任务。Lisa 的团队使用这个区域来放置诸如“评估负载测试工具”或“建立性能测试基线”之类的卡片。珍妮特成功地使用了不同颜色的卡片来表明这个故事是针对从组织其他领域借来的专家角色之一。

Consider a separate row on your story board for tasks needed by the product as a whole. Lisa’s team uses this area to put cards such as “Evaluate load test tools” or “Establish a performance test baseline.” Janet has successfully used different colored cards to show that the story is meant for one of the expert roles borrowed from other areas of the organization.

如果您正在尝试为完整的端到端工作流建立基线,则某些性能测试可能需要等到大部分应用程序构建完成。如果性能和可靠性是重中之重,您需要找到一种方法来在项目早期测试它们。确定故事的优先级,以便钢丝或薄片尽早完成。您应该能够创建一个可以运行的性能测试,并在您向工作流中添加越来越多的功能时继续运行。这可能使您能够及早发现性能问题并重新设计系统架构以进行改进。对于许多应用程序,如果没有必要的性能,正确的功能是无关紧要的。

Some performance tests might need to wait until much of the application is built if you are trying to baseline full end-to-end workflows. If performance and reliability are a top priority, you need to find a way to test those early in the project. Prioritize stories so that a steel thread or thin slice is complete early. You should be able to create a performance test that can be run and continue to run as you add more and more functionality to the workflow. This may enable you to catch performance issues early and redesign the system architecture for improvements. For many applications, correct functionality is irrelevant without the necessary performance.

考虑非功能性测试的时间是在发布或主题规划期间。计划尽早开始,根据需要处理小的增量。对于每次迭代,查看您的团队需要执行哪些任务,以确定代码设计是否可靠、可扩展、可用且安全。在下一节中,我们将研究一些不同类型的象限 4 测试。

The time to think about your nonfunctional tests is during release or theme planning. Plan to start early, tackling small increments as needed. For each iteration, see what tasks your team needs in order to determine whether the code design is reliable, scalable, usable, and secure. In the next section, we’ll look at some different types of Quadrant 4 tests.

“能力”测试

“ility” Testing

如果我们可以只关注应用程序所需的行为和功能,生活就会变得如此简单。不幸的是,我们必须关注安全性、可维护性、互操作性、兼容性、可靠性和可安装性等质量。让我们来看看其中的一些“能力”。

If we could just focus on the desired behavior and functionality of the application, life would be so simple. Unfortunately, we have to be concerned with qualities such as security, maintainability, interoperability, compatibility, reliability, and installability. Let’s take a look at some of these “ilities.”

安全

Security

好的,它并没有以 -ility 结尾,但我们将它包含在“ility”桶中,因为我们使用面向技术的测试来评估产品的安全方面。如今,安全是每个组织的首要任务。每个组织都需要确保其软件的机密性和完整性。他们想要验证诸如不可否认之类的概念,即保证消息已由声称已发送消息的一方发送并由声称已收到消息的一方接收。应用程序需要执行正确的身份验证,确认每个用户的身份和授权,以便只允许用户访问他们有权使用的服务。测试这么多不同的安全方面并不容易。

OK, it doesn’t end in -ility, but we include it in the “ility” bucket because we use technology-facing tests to appraise the security aspects of the product. Security is a top priority for every organization these days. Every organization needs to ensure the confidentiality and integrity of their software. They want to verify concepts such as no repudiation, a guarantee that the message has been sent by the party that claims to have sent it and received by the party that claims to have received it. The application needs to perform the correct authentication, confirming each user’s identity, and authorization, in order to allow the user access only to the services they’re authorized to use. Testing so many different aspects of security isn’t easy.

在急于交付功能时,新成立的组织中的业务专家和开发团队可能不会首先考虑安全性。他们只是想让一些软件工作,这样他们就可以做生意了。授权通常是安全测试的唯一方面,他们将其视为业务功能的一部分。

In the rush to deliver functionality, both business experts and development teams in newly started organizations may not be thinking of security first. They just want to get some software working so they can do business. Authorization is often the only aspect of security testing that they consider as part of business functionality.

丽莎的故事

Lisa’s Story

我现在的团队就是一个很好的例子。该企业对自动化管理 401(k) 计划的功能很感兴趣。他们确实煞费苦心地保护软件和数据,但这不是测试的重点。当我在会议上听到一些关于安全测试的精彩演讲后“信以为真”时,我买了一本关于安全测试的书并开始在网站上四处游荡。我发现了一些严重的问题,我们解决了这些问题,但我们意识到我们需要一种全面的方法来确保安全。我们写故事来实现这一点。我们还开始在每个故事中加入“安全”任务卡,以便我们在开发和测试时注意安全需求。

My current team is a case in point. The business was interested in automating functionality to manage 401(k) plans. They did take pains to secure the software and data, but it wasn’t a testing priority. When I “got religion” after hearing some good presentations about security testing at conferences, I bought a book on security testing and started hacking around on the site. I found some serious issues, which we fixed, but we realized we needed a comprehensive approach to ensuring security. We wrote stories to implement this. We also started including a “security” task card with every story so that we’d be mindful of security needs while developing and testing.

——丽莎

—Lisa

对此类工作进行预算必须是业务优先事项。根据贵公司的优先事项和资源,有多种选择可供选择。在投入大量时间和精力之前了解您的需求和风险。

Budgeting this type of work has to be a business priority. There’s a range of alternatives available, depending on your company’s priorities and resources. Understand your needs and the risks before you invest a lot of time and energy.

珍妮特的故事

Janet’s Story

与我合作的一个团队有一个独立的公司安全团队。每当向应用程序添加可能暴露安全漏洞的功能时,公司团队都会通过安全测试应用程序运行应用程序并为团队生成报告。它使用固定黑盒探针对代码执行静态测试,并暴露了开发人员能够解决的一些薄弱环节。它没有给出应用程序安全级别的总体情况,但这并不是一个主要问题。

One team that I worked with has a separate corporate security team. Whenever functionality is added to the application that might expose a security flaw, the corporate team runs the application through a security test application and produces a report for the team. It performs static testing using a canned black-box probe on the code and has exposed a few weak areas that the developers were able to address. It does not give an overall picture of the security level for the application, but that was not deemed a major concern.

——珍妮特

—Janet

擅长安全测试的测试人员可以执行基于安全风险的测试,这是通过分析架构风险、攻击模式或滥用和误用案例来驱动的。当需要专业技能时,带来你需要的东西,但团队仍然负责确保测试完成。

Testers who are skilled in security testing can perform security risk-based testing, which is driven by analyzing the architectural risk, attack patterns, or abuse and misuse cases. When specialized skills are required, bring in what you need, but the team is still responsible for making sure the testing gets done.

有多种自动化工具可以帮助进行安全验证。静态分析工具可以在不执行应用程序的情况下检查代码,可以检测代码中可能多年不会出现的潜在安全漏洞。实时运行的动态分析工具可以测试 SQL 注入和跨站点脚本等漏洞。知识渊博的安全测试人员进行的手动探索性测试对于检测自动化测试可能遗漏的问题是必不可少的。

There are a variety of automated tools to help with security verification. Static analysis tools, which can examine the code without executing the application, can detect potential security flaws in the code that might not otherwise show up for years. Dynamic analysis tools, which run in real time, can test for vulnerabilities such as SQL injection and cross-site scripting. Manual exploratory testing by a knowledgeable security tester is indispensable to detect issues that automated tests can miss.

仅此对安全测试的简要介绍就说明了为什么专门的培训和工具对于做好它如此重要。对于大多数组织而言,此测试是绝对必需的。一次安全入侵可能足以让一家公司倒闭。即使概率很低,赌注也太高了,不能推迟这些测试。

Just this brief look at security testing shows why specialized training and tools are so important to do a good job of it. For most organizations, this testing is absolutely required. One security intrusion might be enough to take a company out of business. Even if the probability were low, the stakes are too high to put off these tests.

维护成本高昂的代码可能不会像安全漏洞那样迅速扼杀组织的盈利能力,但它可能导致长期、缓慢的死亡。在下一节中,我们将考虑验证可维护性的方法。

Code that costs a lot to maintain might not kill an organization’s profitability as quickly as a security breach, but it could lead to a long, slow death. In the next section we consider ways to verify maintainability.

可维护性

Maintainability

可维护性不是很容易测试的东西。在传统项目中,通常通过使用完整的代码审查或检查来完成。敏捷团队经常使用结对编程,它具有内置的持续代码审查。还有其他方法可以确保代码和测试保持可维护性。

Maintainability is not something that is easy to test. In traditional projects, it’s often done by the use of full code reviews or inspections. Agile teams often use pair programming, which has built-in continual code review. There are other ways to make sure the code and tests stay maintainable.

我们鼓励开发团队制定他们遵循的应用程序代码、测试框架和测试本身的标准和指南。制定自己的标准而不是让其他独立团队制定标准的团队将更有可能遵循这些标准,因为它们对他们有意义。

We encourage development teams to develop standards and guidelines that they follow for application code, the test frameworks, and the tests themselves. Teams that develop their own standards, rather than having them set by some other independent team, will be more likely to follow them because they make sense to them.

我们所说的标准种类包括方法名称或测试名称的命名约定。所有指南都应该易于遵循,并使可维护性更容易。示例是:“成功始终为零,失败必须为负值”、“每个类或模块应该只有一个单一职责”或“所有函数必须是单入口、单出口”。

The kinds of standards we mean include naming conventions for method names or test names. All guidelines should be simple to follow and make maintainability easier. Examples are: “Success is always zero and failure must be a negative value,” “Each class or module should have only one single responsibility,” or “All functions must be single entry, single exit.”

开发 GUI 的标准还使应用程序更易于测试和维护,因为测试人员知道会发生什么并且不需要怀疑行为是对还是错。如果您从 GUI 自动执行测试,它还会增加可测试性。简单的标准,例如“为所有 GUI 对象使用名称而不是默认为计算机分配的标识符”或“页面上不能有两个具有相同名称的字段”帮助团队达到代码可维护的水平,因为为其提供覆盖的自动化测试。

Standards for developing the GUI also make the application more testable and maintainable, because testers know what to expect and don’t need to wonder whether a behavior is right or wrong. It also adds to testability if you are automating tests from the GUI. Simple standards such as, “Use names for all GUI objects rather than defaulting to the computer assigned identifier” or “You cannot have two fields with the same name on a page” help the team achieve a level where the code is maintainable, as are the automated tests that provide coverage for it.

可维护代码支持共享代码所有权。如果所有代码都以相同的风格编写并且团队中的每个人都容易理解,那么程序员从一个领域转移到另一个领域要容易得多。复杂性增加了风险,也使代码更难理解。简单性的 XP 值应该应用于代码。简单的编码标准还可以包括诸如“避免重复——不要复制粘贴方法”之类的指南。这些相同的概念适用于测试框架和测试本身。

Maintainable code supports shared code ownership. It is much easier for a programmer to move from one area to another if all code is written in the same style and easily understood by everyone on the team. Complexity adds risk and also makes code harder to understand. The XP value of simplicity should be applied to code. Simple coding standards can also include guidelines such as, “Avoid duplication—Don’t copy-paste methods.” These same concepts apply to test frameworks and the tests themselves.

可维护性也是自动化测试的一个重要因素。测试工具在使它们易于使用的特性方面落后于编程工具maintain,比如IDE插件,让编写和维护测试脚本更简单、更高效。这种情况变化很快,因此寻找可提供轻松重构和搜索与替换的工具,以及可轻松修改脚本的其他实用程序。

Maintainability is an important factor for automated tests as well. Test tools have lagged behind programming tools in features that make them easy to maintain, such as IDE plug-ins to make writing and maintaining test scripts simpler and more efficient. That’s changing fast, so look for tools that provide easy refactoring and search-and-replace, and for other utilities that make it easy to modify the scripts.

数据库的可维护性也很重要。数据库设计需要灵活和可用。每次迭代都可能带来任务来添加或删除表、列、约束或触发器,或者进行某种数据转换。如果数据库设计不佳或数据库中充斥着无效数据,这些任务就会成为瓶颈。

Database maintainability is also important. The database design needs to be flexible and usable. Every iteration might bring tasks to add or remove tables, columns, constraints, or triggers, or to do some kind of data conversion. These tasks become a bottleneck if the database design is poor or the database is cluttered with invalid data.

丽莎的故事

Lisa’s Story

一个严重的回归错误未被发现并导致生产问题。我们有一个应该已经发现错误的测试。但是,回归套件使用的模式中缺少一个约束。多年来,我们的测试模式随意增长。有些具有生产模式中不再存在的列。有些缺少各种约束、触发器和索引。我们的 DBA 必须根据每个故事的需要手动更改每个模式,而不是在每个模式中运行相同的脚本来更新它。我们在几个冲刺中预算了时间来重新创建所有的测试模式,以便它们是相同的并且也与生产相匹配。

A serious regression bug went undetected and caused production problems. We had a test that should have caught the bug. However, a constraint was missing from the schema used by the regression suite. Our test schemas had grown haphazardly over the years. Some had columns that no longer existed in the production schema. Some were missing various constraints, triggers, and indices. Our DBA had to manually make changes to each schema as needed for each story instead of running the same script in each schema to update it. We budgeted time over several sprints to recreate all of the test schemas so that they were identical and also matched production.

——丽莎

—Lisa

计划时间来评估数据库对团队速度的影响,并像处理生产和测试代码一样重构它。应用程序、测试和执行环境的所有方面的可维护性更多是评估和重构的问题,而不是直接测试。如果你的速度在下降,是因为部分代码难以处理,还是因为数据库难以修改?

Plan time to evaluate the database’s impact on team velocity, and refactor it just as you do production and test code. Maintainability of all aspects of the application, test, and execution environments is more a matter of assessment and refactoring than direct testing. If your velocity is going down, is it because parts of the code are hard to work on, or is it that the database is difficult to modify?

互操作性

Interoperability

互操作性是指不同系统和组织协同工作和共享信息的能力。互操作性测试着眼于两个或多个通信系统之间的端到端功能。这些测试是在用户(人或软件应用程序)的上下文中完成的,并查看功能行为。

Interoperability refers to the capability of diverse systems and organizations to work together and share information. Interoperability testing looks at end-to-end functionality between two or more communicating systems. These tests are done in the context of the user—human or a software application—and look at functional behavior.

在敏捷开发中,互操作性测试可以在开发周期的早期完成。我们在每次迭代结束时都有一个可运行的可部署系统,以便我们可以部署和设置与其他系统的测试。

In agile development, interoperability testing can be done early in the development cycle. We have a working, deployable system at the end of each iteration so that we can deploy and set up testing with other systems.

象限 1 包括代码集成测试,它们是组件之间的测试,但在企业系统中还有另一个级别的集成测试。您可能会发现自己通过开放或专有接口集成系统。您为您的系统开发的 API 可能使您的用户能够轻松地设置一个框架,以便他们轻松地进行测试。为您的客户进行更轻松的测试可以更快地接受。

Quadrant 1 includes code integration tests, which are tests between components, but there is a whole other level of integration tests in enterprise systems. You might find yourself integrating systems through open or proprietary interfaces. The API you develop for your system might enable your users to easily set up a framework for them to test easily. Easier testing for your customer makes for faster acceptance.

第 20 章成功交付”中,我们将更多地讨论这一级别测试的重要性。

In Chapter 20, “Successful Delivery,” we discuss more about the importance of this level of testing.

在 Janet 参与的一个项目中,在客户现场设置了测试系统,以便他们可以尽早开始将它们与自己的系统集成。根据需要更改现有系统的接口,并在每次新部署时进行测试。

In one project Janet worked on, test systems were set up at the customer’s site so that they could start to integrate them with their own systems early. Interfaces to existing systems were changed as needed and tested with each new deployment.

如果你的团队工作的系统必须与外部系统一起工作,你可能无法在你的测试环境中代表它们,除非使用模拟其他系统或设备行为的存根和驱动程序。在这种情况下,开发完成后进行测试可能是不可避免的。您可能必须在多个团队共享的测试环境中安排测试时间。

If the system your team works on has to work together with external systems, you may not be able to represent them all in your test environments except with stubs and drivers that simulate the behavior of the other systems or equipment. This is one situation where testing after development is complete might be unavoidable. You might have to schedule test time in a test environment shared by several teams.

考虑您需要与之通信的所有系统,并确保您提前计划拥有一个合适的环境来一起测试它们。您还需要规划资源来测试您的应用程序是否与可能使用它的各种操作系统、浏览器、客户端、服务器和硬件兼容。接下来我们将讨论兼容性测试。

Consider all of the systems with which yours needs to communicate, and make sure you plan ahead to have an appropriate environment for testing them together. You’ll also need to plan resources for testing that your application is compatible with the various operating systems, browsers, clients, servers, and hardware with which it might be used. We’ll discuss compatibility testing next.

兼容性

Compatibility

您从事的项目类型决定了需要进行多少兼容性测试。如果您有一个 Web 应用程序并且您的客户遍布全球,您将需要考虑所有类型的浏览器和操作系统。如果您要交付自定义企业应用程序,您可能会减少兼容性测试的数量,因为您可能能够决定支持哪些版本。

The type of project you’re working on dictates how much compatibility testing is required. If you have a web application and your customers are worldwide, you will need to think about all types of browsers and operating systems. If you are delivering a custom enterprise application, you can probably reduce the amount of compatibility testing, because you might be able to dictate which versions are supported.

由于每个新屏幕都是作为用户界面故事的一部分开发的,因此最好检查其在所有支持的浏览器中的可操作性。可以将一个简单的任务添加到故事中以在所有浏览器上进行测试。

As each new screen is developed as part of a user interface story, it is a good idea to check its operability in all supported browsers. A simple task can be added to the story to test on all browsers.

珍妮特工作的一个组织必须测试与视障人士阅读软件的兼容性。虽然公司没有正式测试实验室,它在团队区域附近提供测试机器,以便于访问。测试人员定期检查以确保新功能仍然与第三方工具兼容。很容易修复在开发早期发现的问题。

One organization that Janet worked at had to test compatibility with reading software for the visual impaired. Although the company had no formal test lab, it had test machines available near the team area for easy access. The testers made periodic checks to make sure that new functionality was still compatible with the third-party tools. It was easy to fix problems that were discovered early during development.

让测试机器可用于不同的操作系统或浏览器或第三方应用程序,这些应用程序需要与被测系统一起工作,这使测试人员更容易确保与每个新故事或在迭代结束时的兼容性。当您开始一个新的主题或项目时,请考虑您可能需要的资源来验证兼容性。如果您开始开发一种全新的产品,您可能需要为它建立一个测试实验室。确保您的团队获得有关最终用户的硬件、操作系统、浏览器及其版本的信息。如果新浏览器版本的使用百分比增长到足够大,可能是时候开始将该版本纳入兼容性测试了。

Having test machines available with different operating systems or browsers or third-party applications that need to work with the system under test makes it easier for the testers to ensure compatibility with each new story or at the end of an iteration. When you start a new theme or project, think about the resources you might need to verify compatibility. If you’re starting on a brand new product, you might have to build up a test lab for it. Make sure your team gets information on your end users’ hardware, operating systems, browsers, and versions of each. If the percentage of use of a new browser version has grown large enough, it might be time to start including that version in your compatibility testing.

当您选择或创建功能测试工具时,请确保有一种简单的方法可以在不同版本的浏览器、操作系统和硬件上运行相同的脚本。例如,Lisa 的团队可以在 Windows、Solaris 和 Linux 上运行的每台服务器上使用相同的 GUI 回归测试套件。功能测试脚本也可用于可靠性测试。让我们接下来看看。

When you select or create functional test tools, make sure there’s an easy way to run the same script with different versions of browsers, operating systems, and hardware. For example, Lisa’s team could use the same suite of GUI regression tests on each of the servers running on Windows, Solaris, and Linux. Functional test scripts can also be used for reliability testing. Let’s look at that next.

可靠性

Reliability

软件的可靠性可以指系统在常规情况和意外情况下执行和维护其功能的能力。该系统还必须以一致性和可重复性执行和维护其功能。可靠性分析回答了“它会运行多长时间才会坏掉?”这个问题。用于衡量可靠性的一些统计数据是:

Reliability of software can be referred to as the ability of a system to perform and maintain its functions in routine circumstances as well as unexpected circumstances. The system also must perform and maintain its functions with consistency and repeatability. Reliability analysis answers the question, “How long will it run before it breaks?” Some statistics used to measure reliability are:

图像 平均无故障时间:从初始操作到第一次出现故障或故障之间的平均时间。换句话说,系统在第一次出现故障之前可以运行多长时间?

Mean time to failure: The average or mean time between initial operation and the first occurrence of a failure or malfunction. In other words, how long can the system run before it fails the first time?

图像 Mean time between failures(平均故障间隔时间):一种可靠性的统计量度,计算得出预期的平均故障间隔时间。越长越好。

Mean time between failures: A statistical measure of reliability, this is calculated to indicate the anticipated average time between failures. The longer the better.

在传统项目中,我们过去常常安排数周的可靠性测试,试图运行与日常工作相匹配的模拟。现在,我们应该能够在每次迭代结束时交付,那么我们如何安排可靠性测试呢?

In traditional projects, we used to schedule weeks of reliability testing that tried to run simulations that matched a regular day’s work. Now, we should be able to deliver at the end of every iteration, so how can we schedule reliability tests?

我们有定期运行的自动化单元和验收测试。要进行可靠性测试,我们只需要使用相同的测试并一遍又一遍地运行它们。理想情况下,您将使用收集的显示日常使用情况的统计数据,创建一个反映使用情况的脚本,并在您的团队认为足以证明稳定性的稳定构建上运行它。您可以将随机数据输入到测试中以模拟生产使用,并确保应用程序不会因无效输入而崩溃。当然,您可能希望反映高峰使用情况以确保它也能处理繁忙时间。

We have automated unit and acceptance tests running on a regular basis. To do a reliability test, we simply need to use those same tests and run them over and over. Ideally, you would use statistics gathered that show daily usage, create a script that mirrors the usage, and run it on a stable build for however long your team thinks is adequate to prove stability. You can input random data into the tests to simulate production use and make sure the application doesn’t crash because of invalid inputs. Of course, you might want to mirror peak usage to make sure that it handles busy times as well.

您可以在每次迭代中创建故事来开发这些脚本并在将新功能添加到应用程序时添加新功能。您的验收测试可以非常具体,例如“功能 X 必须在 24 小时内至少执行 10,000 次操作,持续至少 3 天。”

You can create stories in each iteration to develop these scripts and add new functionality as it is added to the application. Your acceptance tests could be very specific such as, “Functionality X must perform 10,000 operations in a 24-hour period for a minimum of 3 days.”

当心:运行一千次测试没有任何严重问题并不意味着您拥有可靠的软件。您必须运行正确的测试。要使可靠性测试有效,请考虑您的应用程序以及它在一段时间内全天、每天、如何使用。指定旨在证明您的应用程序即使在高峰时段也能够满足客户需求的测试。

Beware: Running a thousand tests without any serious problems doesn’t mean you have reliable software. You have to run the right tests. To make a reliability test effective, think about your application and how it is used all day, every day, over a period of time. Specify tests that are aimed at demonstrating that your application will be able to meet your customers’ needs, even during peak times.

以可衡量目标的形式向客户团队询问他们的可靠性标准。例如,如果每 10,000 笔交易出现 10 个或更少的错误,或者 Web 应用程序在 99.999% 的时间内可用,他们可能会认为系统是可靠的。从停电和其他灾难中恢复可能是可靠性目标的一部分,并将以服务水平协议的形式说明。知道它们是什么。一些行业有自己的软件可靠性标准和指南。

Ask the customer team for their reliability criteria in the form of measurable goals. For example, they might consider the system reliable if ten or fewer errors occur for every 10,000 transactions, or the web application is available 99.999% of the time. Recovery from power outages and other disasters might be part of the reliability objectives, and will be stated in the form of Service Level Agreements. Know what they are. Some industries have their own software reliability standards and guidelines.

使用正确的程序员和客户测试来推动开发应该会增强应用程序的可靠性,因为这通常会带来更好的设计和更少的缺陷。根据需要编写额外的故事和任务,以交付满足组织可靠性标准的系统。

Driving development with the right programmer and customer tests should enhance the application’s reliability, because this usually leads to better design and fewer defects. Write additional stories and tasks as needed to deliver a system that meets the organization’s reliability standards.

您的产品在启动和运行后可能是可靠的,但它也需要在所有受支持的环境中可供所有用户安装。这是遵循敏捷原则给我们带来优势的另一个领域。

Your product might be reliable after it’s up and running, but it also needs to be installable by all users, in all supported environments. This is another area where following agile principles gives us an advantage.

安装性

Installability

成功的敏捷团队的基石之一是持续集成。这意味着构建可以在一天中的任何时间进行测试。许多团队选择每天将一个或多个成功的构建部署到测试环境中。

One of the cornerstones of a successful agile team is continuous integration. This means that a build is ready for testing anytime during the day. Many teams choose to deploy one or more of the successful builds into test environments on a daily basis.

自动化部署创建可重复性并使部署成为非事件。这让我们很兴奋,因为我们已经经历了数周的时间来尝试集成和安装一个新系统。我们知道,如果我们构建一次并将相同的构建部署到多个环境,我们就已经形成了一致性。

Automating the deployment creates repeatability and makes deployment a non-event. This is exciting to us because we have experienced weeks of trying to integrate and install a new system. We know that if we build once and deploy the same build to multiple environments, we have developed consistency.

珍妮特的故事

Janet’s Story

在我参与的一个项目中,部署是自动进行的,并在开发周期的多个环境中进行了测试。但是,在部署到客户站点时出现了问题。我们在结束游戏中添加了一个步骤,以便支持小组可以发布并进行完整的安装测试,就好像它是客户的站点一样。我们能够浏览部署说明并消除客户原本会看到的许多问题。

On one project I worked on, the deployment was automatic and was tested on multiple environments in the development cycle. However, there were issues when deploying to the customer site. We added a step to the end game so that the support group would take the release and do a complete install test as if it were the customer’s site. We were able to walk through the deployment notes and eliminated many of the issues the customer would have otherwise seen.

——珍妮特

—Janet

与任何其他功能一样,需要评估与安装相关的风险并相应地确定测试量。我们的建议是尽早并经常这样做,并尽可能自动化该过程。

As with any other functionality, risks associated with installation need to be evaluated and the amount of testing determined accordingly. Our advice is to do it early and often, and automate the process if possible.

第 20 章成功交付”更多地介绍了安装测试。

Chapter 20, “Successful Delivery,” has more on installation testing.

“能力”总结

“ility” Summary

根据您的产品领域,还有其他“能力”需要测试。安全关键软件,例如用于医疗设备和飞机控制系统的软件,需要进行广泛的安全测试,而回归测试可能会包含与安全相关的测试。系统冗余和故障转移测试对于此类产品尤为重要。您的团队可能需要查看有关软件相关安全问题的行业数据并使用额外的代码审查。可配置性、可审核性、可移植性、健壮性和可扩展性只是您的团队可能需要通过面向技术的测试来评估的几个品质。

There are other “ilities” to test, depending on your product’s domain. Safety-critical software, such as that used in medical devices and aircraft control systems, requires extensive safety testing, and the regression tests probably would contain tests related to safety. System redundancy and failover tests would be especially important for such a product. Your team might need to look at industry data around software-related safety issues and use extra code reviews. Configurability, auditability, portability, robustness, and extensibility are just a few of the qualities your team might need to evaluate with technology-facing tests.

无论您需要测试什么“能力”,请使用增量方法。首先引出客户团队的要求和他们针对特定质量领域的目标示例。编写面向业务的测试以确保代码旨在满足这些目标。在第一次迭代中,团队可能会做一些研究并提出测试策略来评估产品的现有质量水平。下一步可能是创建一个合适的测试环境,研究工具,或者从一些手动测试开始。

Whatever “ility” you need to test, use an incremental approach. Start by eliciting the customer team’s requirements and examples of their objectives for that particular area of quality. Write business-facing tests to make sure the code is designed to meet those goals. In the first iteration, the team might do some research and come up with a test strategy to evaluate the existing quality level of the product. The next step might be to create a suitable test environment, to research tools, or to start with some manual tests.

当您了解应用程序如何满足客户的要求时,使用新的象限 1 和 2 测试关闭循环,使应用程序更接近该特定属性的目标。还建议对下一节中介绍的性能、负载和其他测试使用增量方法。

As you learn how the application measures up to the customers’ requirements, close the loop with new Quadrant 1 and 2 tests that drive the application closer to the goals for that particular property. An incremental approach is also recommended for performance, load, and other tests that are addressed in the next section.

性能、负载、压力和可扩展性测试

Performance, Load, Stress, and Scalability Testing

性能、负载、压力和可伸缩性测试都属于第 4 象限,因为它们的技术重点。通常需要专业技能,尽管许多团队已经想出了在这些领域进行自己测试的方法。我们先谈谈可扩展性,因为它经常被遗忘。

Performance, load, stress, and scalability testing all fall into Quadrant 4 because of their technology focus. Often specialized skills are required, although many teams have figured out ways to do their own testing in these areas. Let’s talk about scalability first, because it is often forgotten.

可扩展性

Scalability

可扩展性测试验证应用程序在添加更多用户时仍然可靠。这真正的意思是,“您的系统能否处理不断增长的客户群的容量?” 听上去很简单,其实不然,是一个敏捷团队通常无法独自解决的问题。

Scalability testing verifies the application remains reliable when more users are added. What that really means is, “Can your system handle the capacity of a growing customer base?” It sounds simple, but really isn’t, and is a problem that an agile team usually can’t solve by itself.

重要的是要考虑整个系统,而不仅仅是应用程序本身。例如,网络往往是瓶颈,因为它无法处理增加的吞吐量。数据库呢?它会扩展吗?您正在使用的硬件会处理正在考虑的新负载吗?只是添加新硬件很简单,还是瓶颈?

It is important to think about the whole system and not just the application itself. For example, the network is often the bottleneck, because it can’t handle the increased throughput. What about the database? Will it scale? Will the hardware you are using handle the new loads being considered? Is it simple just to add new hardware, or is it the bottleneck?

珍妮特的故事

Janet’s Story

在我最近工作的一个组织中,他们的客户群增长非常快,他们投资的解决方案由于硬件限制已经达到了它的容量。添加新服务器不是一件简单的事情,因为解决方案不是那样设计的。需要监控系统以在高峰使用期间重新启动服务。

In one organization I was recently working in, their customer base had grown very quickly, and the solution they had invested in had reached its capacity due to hardware constraints. It was not a simple matter of adding a new server, because the solution was not designed that way. The system needed to be monitored to restart services during peak usage.

为了发展,该组织实际上必须改变解决方案以适应其未来的发展,但直到问题开始发生时才意识到这一点。

To grow, the organization had to actually change solutions to accommodate its future growth, but this was not recognized until problems started to happen.

理想情况下,组织会在旧系统出现问题之前更换旧系统。这是为什么了解您的系统及其功能以及未来增长预测很重要的示例。

Ideally, the organization would have replaced the old system before it was an issue. This is an example of why it is important to understand your system and its capability, as well as future growth projections.

——珍妮特

—Janet

您需要走出团队才能获得解决可扩展性问题所需的答案,因此请提前计划。

You will need to go outside the team to get the answers you require to address scalability issues, so plan ahead.

性能和负载测试

Performance and Load Testing

性能测试通常用于帮助识别系统中的瓶颈或为将来的测试建立基线。这样做也是为了确保符合性能目标和要求,并帮助利益相关者做出与被测试应用程序的整体质量相关的明智决策。

Performance testing is usually done to help identify bottlenecks in a system or to establish a baseline for future testing. It is also done to ensure compliance with performance goals and requirements, and to help stakeholders make informed decisions related to the overall quality of the application being tested.

随着越来越多的用户同时访问系统,负载测试会评估系统行为。压力测试评估应用程序在高于预期的负载下的健壮性。应用程序会随着业务的增长而扩展吗?对于某些应用程序,响应时间等特性可能比功能更为重要。

Load testing evaluates system behavior as more and more users access the system at the same time. Stress testing evaluates the robustness of the application under higher-than-expected loads. Will the application scale as the business grows? Characteristics such as response time can be more critical than functionality for some applications.

Grig Gheorghiu [ 2005 ] 强调需要明确定义的期望才能从性能测试中获得价值。他说,“如果你不知道在系统方面你想去哪里,那么你选择哪个方向就无关紧要了(还记得爱丽丝和柴郡猫吗?)。” 例如,您可能想知道 Web 应用程序的并发用户数和可接受的响应时间。

Grig Gheorghiu [2005] emphasizes the need for clearly defined expectations to get value from performance testing. He says, “If you don’t know where you want to go in terms of the system, then it matters little which direction you take (remember Alice and the Cheshire Cat?).” For example, you probably want to know the number of concurrent users and the acceptable response time for a web application.

性能和负载测试工具

Performance and Load-Testing Tools

定义性能目标后,您可以使用各种工具对系统施加负载并检查瓶颈。这可以在单元级别完成,使用 JUnitPerf、httperf 或自制工具等工具。Apache JMeter、The Grinder、Pounder、ftptt 和 OpenWebLoad 是撰写本文时可用的众多开源性能和负载测试工具的更多示例。其中一些,例如 JMeter,可用于多种服务器类型,从SOAP到 LDAP 再到 POP3 邮件。许多商业工具选项也可用,包括 NeoLoad、WebLoad、eValid LoadTest、LoadRunner 和 SOATest。

After you’ve defined your performance goals, you can use a variety of tools to put a load on the system and check for bottlenecks. This can be done at the unit level, with tools such as JUnitPerf, httperf, or a home-grown harness. Apache JMeter, The Grinder, Pounder, ftptt, and OpenWebLoad are more examples of the many open source performance and load test tools available at the time of this writing. Some of these, such as JMeter, can be used on a variety of server types, from SOAP to LDAP to POP3 mail. Plenty of commercial tool options are available too, including NeoLoad, WebLoad, eValid LoadTest, LoadRunner, and SOATest.

请参阅参考书目以获取指向您可以研究工具的站点的链接。

See the bibliography for links to sites where you can research tools.

使用这些工具来查找性能瓶颈。Lisa 的团队使用 JProfiler 查找应用程序瓶颈和内存泄漏,并使用 JConsole 分析数据库使用情况。.NET 和其他环境存在类似的工具,包括 .NET Memory Profiler 和 ANTS Profiler Pro。正如 Grig 指出的那样,有特定于数据库的分析器可以查明数据库级别的性能问题;请您的数据库专家与您合作。您的系统管理员可以帮助您使用 shell 命令(例如 top)或工具(例如 PerfMon)来监控 CPU、内存、交换区、磁盘 I/O 和其他硬件资源。类似的工具在网络级别可用,例如 NetScout。

Use these tools to look for performance bottlenecks. Lisa’s team uses JProfiler to look for application bottlenecks and memory leaks, and JConsole to analyze database usage. Similar tools exist for .NET and other environments, including .NET Memory Profiler and ANTS Profiler Pro. As Grig points out, there are database-specific profilers to pinpoint performance issues at the database level; ask your database experts to work with you. Your system administrators can help you use shell commands such as top, or tools such as PerfMon to monitor CPU, memory, swap, disk I/O, and other hardware resources. Similar tools are available at the network level, for example, NetScout.

您还可以使用团队最熟悉的工具。在一个项目中,Janet 与其中一位程序员密切合作来创建测试。她帮助他根据客户的性能和负载期望定义所需的测试,然后他使用 JUnit 将这些测试自动化。他们一起分析结果并向客户报告。

You can also use the tools the team is most familiar with. In one project, Janet worked very closely with one of the programmers to create the tests. She helped him to define the tests needed based on customer’s performance and load expectations, and he automated them using JUnit. Together they analyzed the results to report back to the customer.

建立基线是评估绩效的良好开端。下一节将探讨性能测试的这一方面。

Establishing a baseline is a good first step for evaluating performance. The next section explores this aspect of performance testing.

基线

Baseline

性能调整可能会变成一个大项目,因此必须提供一个基准,您可以将其与新版本软件的性能进行比较。即使目前性能不是您最关心的问题,也不要忽视它。获得性能基线是个好主意,这样您以后就知道您的响应时间将朝哪个方向发展。Lisa 的公司托管了一个负载很小的网站。他们在站点上获得了负载测试基线,以便随着站点的增长,他们知道性能受到了怎样的影响。

Performance tuning can turn into a big project, so it is essential to provide a baseline that you can compare against new versions of the software on performance. Even if performance isn’t your biggest concern at the moment, don’t ignore it. It’s a good idea to get a performance baseline so that you know later which direction your response time is headed. Lisa’s company hosts a website that has had a small load on it. They got a load test baseline on the site so that as it grew, they’d know how performance was being affected.

如果已经为特定功能定义了特定的性能标准,我们建议将性能测试作为迭代的一部分进行,以确保在为时已晚而无法解决问题之前发现问题。

If there are specific performance criteria that have been defined for specific functionality, we suggest that performance testing be done as part of the iteration to ensure that issues are found before it is too late to fix them.

基准测试可以在发布期间的任何时间进行。如果添加了可能影响性能的新功能,例如复杂的查询,请重新运行测试以确保没有不利影响。这样,当开发团队仍然熟悉该功能时,您就有时间在周期的早期优化查询或代码。

Benchmarking can be done at any time during a release. If new functionality is added that might affect the performance, such as complicated queries, rerun the tests to make sure there are no adverse effects. This way, you have time to optimize the query or code early in the cycle when the development team is still familiar with the feature.

除非在模拟生产环境的环境中运行,否则任何性能、负载或压力测试都没有意义。让我们多谈谈环境。

Any performance, load, or stress test won’t be meaningful unless it’s run in an environment that mimics the production environment. Let’s talk more about environments.

测试环境

Test Environments

性能测试的最终运行将帮助客户做出接受其产品的决定。为了获得准确的结果,需要在与生产设备相似的设备上运行测试。团队通常会使用较小的机器并推断结果以确定性能是否足以满足业务需求。报告测试结果时应明确指出这一点。

Final runs of the performance tests will help customers make decisions about accepting their product. For accurate results, tests need to be run on equipment that is similar to that of production. Often teams will use smaller machines and extrapolate the results to decide if the performance is sufficient for the business needs. This should be clearly noted when reporting test results.

也可以在发布期间的任何时间对应用程序施加压力以查看它在崩溃之前可以承受的负载,但通常客户不会将其视为高优先级,除非您的任务关键型系统具有大量负载。

Stressing the application to see what load it can take before it crashes can also be done anytime during the release, but usually it is not considered high-priority by customers unless you have a mission-critical system with lots of load.

受负载增加影响的一种资源是内存。在下一节中,我们将讨论内存管理。

One resource that is affected by increasing load is memory. In the next section, we discuss memory management.

内存管理

Memory Management

内存通常根据用于 RAM、ROM、硬盘驱动器等的内存量(通常是最小值或最大值)来描述。您应该了解内存使用情况并注意是否存在泄漏,因为当应用程序在使用高峰期间处于生产状态时,它们可能会导致灾难性故障。一些编程语言更容易受到内存问题的影响,因此了解代码的优缺点将帮助您了解要注意什么。内存问题测试可以作为性能、负载和压力测试的一部分来完成。

Memory is usually described in terms of the amount (normally the minimum or maximum) of memory to be used for RAM, ROM, hard drives, and so on. You should be aware of memory usage and watch for leaks, because they can cause catastrophic failures when the application is in production during peak usage. Some programming languages are more susceptible to memory issues, so understanding the strengths and weaknesses of the code will assist you in knowing what to watch for. Testing for memory issues can be done as part of performance, load, and stress testing.

垃圾收集是一种用于将内存释放回程序的工具。但是,它可以掩盖严重的内存问题。如果您看到可用内存随着使用量稳定下降,然后突然增加到最大可用内存,您可能怀疑垃圾收集已经启动。观察模式中的异常情况或系统是否在大量使用下开始变慢。您可能需要监视一段时间并与程序员一起查找问题。修复可能很简单,例如更频繁地安排垃圾收集或将触发级别设置得更高。

Garbage collection is one tool used to release memory back to the program. However, it can mask severe memory issues. If you see the available memory steadily decreasing with usage and then all of a sudden increasing to maximum available, you might suspect the garbage collection has kicked in. Watch for anomalies in the pattern or whether the system starts to get slow under heavy usage. You may need to monitor for a while and work with the programmers to find the issue. The fix might be something simple, such as scheduling the garbage collection more often or setting the trigger level higher.

当您与程序员一起编写故事时,询问他们是否预计会出现记忆问题。如果您知道该地区可能存在风险,您可以专门进行测试。观察内存泄漏并不总是那么容易,但有一些工具可以提供帮助。这是程序员应该拥有易于使用的工具的领域。与他们合作以验证应用程序没有内存问题。执行上一节中描述的性能和负载测试以验证没有任何内存问题。

When you are working with the programmers on a story, ask them if they expect problems with memory. You can test specifically if you know there might be a risk in the area. Watching for memory leaks is not always easy, but there are tools to help. This is an area where programmers should have tools easily available. Collaborate with them to verify that the application is free of memory issues. Perform the performance and load tests described in the previous section to verify that there aren’t any memory problems.

您不必是如何进行面向技术的测试来批评产品以帮助您的团队计划和执行它的专家。您的团队可以从该象限评估需要进行哪些测试。在计划发布时谈论这些测试;如果您以前没有做过,您可以专门针对性能和负载创建一个测试计划。您将需要时间来获得所需的专业知识,方法是通过识别和学习技能来获得它,或者通过引入外部帮助来获得。与所有开发工作一样,将面向技术的测试分解为可以在每次迭代中解决和构建的小任务。

You don’t have to be an expert on how to do technology-facing testing that critiques the product to help your team plan for it and execute it. Your team can evaluate what tests it needs from this quadrant. Talk about these tests as you plan your release; you can create a test plan specifically for performance and load if you’ve not done it before. You will need time to obtain the expertise needed, either by acquiring it through identifying and learning the skills, or by bringing in outside help. As with all development efforts, break technology-facing tests into small tasks that can be addressed and built upon each iteration.

概括

Summary

在本章中,我们探索了第四个敏捷测试象限,即评判产品的面向技术的测试。

In this chapter, we’ve explored the fourth agile testing quadrant, the technology-facing tests that critique the product.

图像开发团队应评估其是否拥有或能够获得进行这些测试的专业知识,或者是否需要计划引入外部资源。

The developer team should evaluate whether it has, or can acquire, the expertise to do these tests, or if it needs to plan to bring in external resources.

图像这些测试采用增量方法,在每次迭代中完成任务,确保有时间解决出现的任何问题并避免生产问题。

An incremental approach to these tests, completing tasks in each iteration, ensures time to address any issues that arise and avoid production problems.

图像 团队应考虑各种类型的“能力”测试,包括安全性、可维护性、互操作性、兼容性、可靠性和可安装性测试,并应在适当的时间执行这些测试。

The team should consider various types of “ility” testing, including security, maintainability, interoperability, compatibility, reliability, and installability testing, and should execute these tests at appropriate times.

图像性能、可伸缩性、压力和负载测试应该从项目一开始就完成。

Performance, scalability, stress, and load testing should be done from the beginning of the project.

图像研究可能影响您的产品的内存管理问题,并计划测试以验证应用程序没有内存问题。

Research the memory management issues that might impact your product, and plan tests to verify the application is free of memory issues.

第 12 章测试象限总结

Chapter 12 Summary of Testing Quadrants

图像

第 6 章中,我们介绍了测试象限,在接下来的章节中,我们讨论了如何在敏捷项目中使用这些概念。在本章中,我们将通过一个使用来自所有四个象限的测试的敏捷团队的示例将它们整合在一起。

In Chapter 6, we introduced the testing quadrants, and in the chapters that followed we talked about how to use the concepts in your agile project. In this chapter, we’ll bring it all together with an example of an agile team that used tests from all four quadrants.

测试象限审查

Review of the Testing Quadrants

我们刚刚用五章讨论了每个象限(参见图 12-1)和可用于不同类型测试的工具示例。下一个技巧是了解您的项目需要哪些测试以及何时进行。在本章中,我们将带您了解一个敏捷项目的真实示例,该项目使用了来自所有四个敏捷测试象限的测试。

We’ve just spent five chapters talking about each of the quadrants (see Figure 12-1) and examples of tools you can use for the different types of testing. The next trick is to know which tests your project needs and when to do them. In this chapter, we’ll walk you through a real-life example of an agile project that used tests from all four agile testing quadrants.

图 12-1敏捷测试象限

Figure 12-1 Agile Testing Quadrants

图像

系统测试示例

A System Test Example

下面的故事是关于一个组织使用各种本土和开源工具成功测试其整个系统的故事。Janet 与这个团队一起工作,Paul Rogers 是主要的测试架构师。这是保罗的故事。

The following story is about one organization’s success in testing its whole system using a variety of home-grown and open source tools. Janet worked with this team, and Paul Rogers was the primary test architect. This is Paul’s story.

应用程序

The Application

该系统解决了远程油气生产井的监控问题。该解决方案结合了一个远程监控设备,该设备可以通过卫星通信信道使用专有协议传输数据并从中央监控站接收调整。

The system solves the problem of monitoring remote oil and gas production wells. The solution combines a remote monitoring device that can transmit data and receive adjustments from a central monitoring station using a proprietary protocol over a satellite communication channel.

远程数据监控系统架构如图12-2所示。油井上的测量设备,远程终端单元 (RTU),使用多种与测量设备通信的协议。来自每个 RTU 的数据通过卫星传输到位于客户总部的服务器。然后通过 Web 界面将其提供给用户。当特定读数超出正常操作限制时,可以通过电子邮件、传真或电话通知系统。Java消息服务 (JMS)提要和 Web 服务也可用于帮助与客户的其他应用程序集成。

Figure 12-2 shows the architecture of the Remote Data Monitoring system. The measurement devices on the oil wells, Remote Terminal Units (RTU), use a variety of protocols to communicate with the measurement device. This data from each RTU is transmitted via satellite to servers located at the client’s main office. It is then made available to users via a web interface. A notification system, via email, fax, or phone, is available when a particular reading is outside of normal operational limits. A Java Message Service (JMS) feed and web services are also available to help integration with clients’ other applications.

图12-2远程数据监控系统架构

Figure 12-2 Remote data monitoring system architecture

图像

该软件应用程序是一个巨大的遗留系统,几乎没有单元测试。该团队正在使用新技术慢慢重建应用程序。

The software application was a huge legacy system that had few unit tests. The team was slowly rebuilding the application with new technology.

团队和流程

The Team and the Process

该团队由四名软件程序员、两名固件程序员、三到四名测试人员、一名产品工程师和一名场外经理组成。“真正的”客户在另一个国家。开发组使用XP实践,包括结对编程和 TDD。客户团队对积压工作使用缺陷跟踪系统,但故事的大部分可见性是通过索引卡。在迭代计划会议期间使用故事卡,任务板跟踪进度。

The team consisted of four software programmers, two firmware programmers, three to four testers, a product engineer, and an off-site manager. The “real” customer was in another country. The development team uses XP practices, including pair programming and TDD. The customer team used the defect-tracking system for the backlog, but most of the visibility of the stories was through index cards. Story cards were used during iteration planning meetings, and the task board tracked the progress.

Scrum 被用作组织和客户的外部报告机制。该团队进行了两周的迭代,大约每四个月发布一次产品。这取决于正在开发的功能。回顾是每次迭代计划会议的一部分,并对讨论的前三个优先事项采取行动。

Scrum was used as the outside reporting mechanism to the organization and the customers. The team had two week iterations and released the product about every four months. This varied depending on the functionality being developed. Retrospectives were held as part of every iteration planning session, and action was taken on the top three priority items discussed.

通过 CruiseControl 的持续集成为测试人员提供了持续的构建,并在每次迭代结束时进行了演示。每个测试人员都有一个用于测试 Web 应用程序的本地环境,但是系统可以使用三个测试环境。第一个是测试新故事,并根据需要使用最新版本进行了更新。第二个用于测试客户报告的问题,因为它已向客户发布了最后一个版本。第三个环境是一个完整的独立测试环境,可用于测试完整部署、通信链路以及固件和硬件。正是在这个环境中,我们运行了负载和可靠性测试。

Continuous integration through CruiseControl provided constant builds for the testers and the demonstrations held at the end of every iteration. Each tester had a local environment for testing the web application, but there were three test environments available to the system. The first one was to test new stories and was updated as needed with the latest build. The second one was for testing client-reported issues, because it had the last version released to the clients. The third environment was a full stand-alone test environment that was available for testing full deploys, communication links, and the firmware and hardware. It was on this environment that we ran our load and reliability tests.

测试驱动开发

Tests Driving Development

驱动开发的测试包括单元测试和验收测试。

The tests driving development included unit test and acceptance tests.

单元测试

Unit Tests

单元测试是支持编程的面向技术的测试。那些作为测试驱动开发的一部分而开发的,不仅可以帮助程序员正确理解故事,还可以帮助设计系统。

Unit tests are technology-facing tests that support programming. Those that are developed as part of test-driven development not only help the programmer get the story right but also help to design the system.

第 7 章,“支持团队的面向技术的测试”,详细解释了单元测试和 TDD。

Chapter 7, “Technology-Facing Tests that Support the Team,” explains more about unit testing and TDD.

远程数据监控项目的程序员全心投入测试驱动开发 (TDD) 和结对编程。所有新功能都是使用结对编程开发和测试的。所有交付给测试人员的故事都得到了单元测试的支持,编码完成后发现的错误很少。发现的错误通常与集成相关。

The programmers on the Remote Data Monitoring project bought into Test Driven Development (TDD) and pair programming wholeheartedly. All new functionality was developed and tested using pair programming. All stories delivered to the testers were supported by unit tests, and very few bugs were found after coding was complete. The bugs that were found were generally integration-related.

然而,当团队刚开始时,遗留系统几乎没有单元测试来支持重构。随着流程变更的实施,开发人员决定开始解决这个问题。每次他们接触遗留系统中的一段代码时,他们都会添加单元测试并根据需要重构代码。渐渐地,遗留系统变得更加稳定,并且能够在需要时承受重大重构。我们体验了单元测试的力量!

However, when the team first started, the legacy system had few unit tests to support refactoring. As process changes were implemented, the developers decided to start fixing the problem. Every time they touched a piece of code in the legacy system, they added unit tests and refactored the code as necessary. Gradually, the legacy system became more stable and was able to withstand major refactoring when it was needed. We experienced the power of unit tests!

验收测试

Acceptance Tests

产品工程师(客户代理人)负责创建验收测试。这些测试的形式因实际情况而异。尽管一开始他很挣扎,但产品工程师非常擅长在程序员开始编码之前给他们进行测试。该团队创建了一个测试模板,它随着时间的推移而发展,同时满足了程序员和测试人员的需求。

The product engineer (the customer proxy) took ownership of creating the acceptance tests. These tests varied in format depending on the actual story. Although he struggled at first, the product engineer got pretty good at giving the tests to the programmers before they started coding. The team created a test template, which evolved over time, that met both the programmers’ and the testers’ needs.

测试有时是非正式编写的,但它们包括数据、需要的设置(如果不是很明显的话)、对故事至关重要的不同变体以及一些示例。该团队发现示例有助于阐明对许多故事的期望。

The tests were sometimes informally written, but they included data, required setup if it wasn’t immediately obvious, different variations that were critical to the story, and some examples. The team found that examples helped clarify the expectations for many of the stories.

测试团队尽快将验收测试自动化,通常是在开发故事的同时进行。当然,产品工程师可以回答开发过程中出现的任何问题。

The test team automated the acceptance tests as soon as possible, usually at the same time as the stories were being developed. Of course, the product engineer was available to answer any questions that came up during development.

这些验收测试有三个目的。它们是支持开发的面向业务的测试,因为它们是在编码开始之前提供给团队的。其次,测试团队将它们用作回归套件的自动化基础,并为探索性测试提供了未来的想法。第三个目的是确认实施满足了客户的需求。产品工程师做了这个方案验证。

These acceptance tests served three purposes. They were business-facing tests that supported development because they were given to the team before coding started. Secondly, they were used by the test team as the basis of automation that fed into the regression suite and provided future ideas for exploratory testing. The third purpose was to confirm that the implementation met the needs of the customer. The product engineer did this solution verification.

请参阅第 8 章,“支持团队的面向业务的测试”,了解有关使用验收测试推动开发的更多信息。

See Chapter 8, “Business-Facing Tests that Support the Team,” for more about driving development with acceptance tests.

自动化

Automation

自动化涉及功能测试结构、Web 服务和嵌入式测试。

Automation involved the functional test structure, web services, and embedded testing.

自动化功能测试结构

The Automated Functional Test Structure

Ruby 与 Watir 一起用作功能自动化框架的首选工具。它被确定为被测系统所需的定制具有最大的灵活性和机会。

Ruby was used with Watir as the tool of choice for the functional automation framework. It was determined to have the most flexibility and opportunity for customization that was required for the system under test.

自动化测试代码包括三个不同的层,如图12-3所示。最低层,第 1 层,包括 Watir 和其他类,例如写入日志文件的记录器。

The automated test code included three distinct layers, shown in Figure 12-3. The lowest layer, Layer 1, included Watir and other classes, such as loggers that wrote to the log files.

图 12-3功能测试层

Figure 12-3 Functional test layers

图像

第二层,第 2 层,是页面访问层,其中包含用于访问各个网页的代码的类。例如,在被测应用程序 (AUT) 中有一个登录页面、一个创建用户页面和一个编辑用户页面。用 Ruby 编写的类包含可以在 AUT 中执行某些功能的代码,例如登录应用程序的类、编辑用户的类以及为用户分配访问权限的类。这些类不包含任何数据。例如,登录类不知道用什么用户名登录。

The second layer, Layer 2, was the page access layer, where classes that contained code to access individual web pages lived. For example, in the application under test (AUT) there was a login page, a create user page, and an edit user page. Classes written in Ruby contained code that could perform certain functions in the AUT, such as a class that logs into the application, a class to edit a user, and a class to assign access rights to a user. These classes contained no data. For example, the log-in class didn’t know what username to log in with.

第三层也是顶层,第 3 层,是测试层,它包含执行测试所需的数据。它称为第 2 层类,后者又称为第 1 层。

The third and top layer, Layer 3, was the test layer, and it contained the data needed to perform a test. It called Layer 2 classes, which in turned called Layer 1.

例如,实际测试将调用 LogIn 并传递 Janet 作为用户名和 Passw0rd 作为密码。这意味着您可以轻松地输入许多不同的数据集。

For example, the actual test would call LogIn and pass Janet as the user name and Passw0rd as the password. This meant you could feed in many different data sets easily.

LogIn ('Janet', 'Passw0rd')

LogIn ('Janet', 'Passw0rd')

第 2 层还知道如何处理应用程序生成的错误消息。例如,当在登录页面上输入了无效的用户名时,登录类会检测到错误消息,然后将问题传递回第 3 层中的测试。

Layer 2 also knew how to handle the error messages the application generated. For example, when an invalid username was entered on the login page, the login class detected the error message and then passed the problem back to the tests in Layer 3.

这意味着相同的第 2 层类可用于快乐路径测试和负面测试。在否定的情况下,第 3 层会期望第 2 层返回失败,然后通过访问第 2 层从浏览器中抓取的错误消息来检查测试失败的原因是否正确。

This means the same Layer 2 classes could be used for both happy path testing and for negative testing. In the negative case, Layer 3 would expect Layer 2 to return a failure, and would then check to see if the test failed for the correct reason by accessing the error messages that Layer Two scraped from the browser.

功能测试使用 Ruby 和 Watir 来控制浏览器上的 DOM,并且可以访问页面中几乎所有的对象。自动化测试套件在每晚构建时运行,以向团队提供有关高级应用程序行为的一致反馈。当团队继续构建单元测试时,这是一个救星。该架构有效地适应了支持团队的面向业务的测试。

The functional tests used Ruby with Watir to control the DOM on the browser and could access almost all of the objects in the page. The automated test suite was run on nightly builds to give the team consistent feedback on high-level application behavior. This was a lifesaver as the team continued to build out the unit tests. This architecture efficiently accommodated the business-facing tests that support the team.

网页服务

Web Services

客户端使用 Web 服务来与他们的一些其他应用程序进行交互。开发组使用 Ruby 编写了一个客户端来测试他们开发的每个服务。对于这些测试,使用了 Ruby 的单元测试框架 Test::Unit。

Web services were used by clients to interface with some of their other applications. The development group used Ruby to write a client to test each service they developed. For these tests, Ruby’s unit testing framework, Test::Unit, was used.

测试团队扩展了 Web 服务测试以涵盖 1,000 多个不同的测试用例,并且仅需几分钟即可运行。他们在短时间内为团队提供了大量的报道。

The web services tests were expanded by the test team to cover more than 1,000 different test cases, and took just minutes to run. They gave the team an amazing amount of coverage in a short period of time.

该团队向客户展示了测试客户端,客户也决定使用它。然而,客户随后认为它对他们不起作用,所以他们开始编写自己的测试,尽管使用 Ruby 以一种更加临时的方式进行。

The team demonstrated the test client to the customers, who decided to use it as well. However, the customers subsequently decided it didn’t work for them, so they started writing their own tests, albeit in a much more ad hoc fashion using Ruby.

他们使用 IRB,Ruby 提供的交互式界面,并以探索性方法提供值。它为客户提供了一个交互式环境,用于发现哪些有效,哪些无效。它还让他们熟悉了 Ruby 以及我们的测试方式,这让他们对我们的测试更有信心。他们的大部分用户验收测试都是使用 IRB 完成的。

They used IRB, the interactive interface provided by Ruby, and fed values in an exploratory method. It gave the customer an interactive environment for discovering what worked and what didn’t. It also let them get familiar with Ruby and how we were testing, and it gave them much more confidence in our tests. Much of their User Acceptance Testing was done using IRB.

Web 服务测试的三种不同倾向服务于三个不同的目的。程序员用它来帮助测试他们的客户端并推动他们的开发。测试人员使用它以非常高效的自动化方式对产品进行评判方式,并且客户能够使用 IRB 测试交付给他们的 Web 服务。

Three different slants on the web services tests served three different purposes. The programmers used it to help test their client and drive their development. The testers used it to critique the product in a very efficient automated manner, and the customers were able to test the web services delivered to them using IRB.

嵌入式测试

Embedded Testing

除了 Web 界面外,RDM 系统还包含一个小型嵌入式设备,该设备使用各种协议与测量设备进行通信。使用 Ruby,开发了各种测试来测试其管理界面的一部分。该界面是类似于 FTP 的命令行系统。

In addition to the web interface, the RDM system consisted of a small embedded device that communicated with measuring equipment using various protocols. Using Ruby, various tests were developed to test part of its administrative interface. This interface was a command-line system similar to FTP.

这些数据驱动的测试包含在 Excel 电子表格中。Ruby 脚本将使用 OLE 接口从 Excel 中读取命令并将它们发送到嵌入式设备。然后脚本会将设备的响应与电子表格中保存的预期结果进行比较。错误以红色突出显示。这些自动化测试大约需要一个小时才能运行,而手动执行相同的测试则需要八个小时。

These data-driven tests were contained in an Excel spreadsheet. A Ruby script would read commands from Excel using the OLE interface and send them to the embedded device. The script would then compare the response from the device with the expected result, also held in the spreadsheet. Errors were highlighted in red. These automated tests took approximately one hour to run, while doing the same tests manually would take eight hours.

虽然这提供了很多测试范围,但实际上并没有测试使用设备的原因,即从 RTU 读取数据。模拟器是用 Ruby 编写的,带有 FOX (FXRuby) GUI。这允许将模拟数据馈送到设备中。由于模拟器可以远程控制,它被纳入自动化测试中,以锻炼嵌入式设备读取数据、响应错误条件以及在输入数据超过预定阈值时生成警报的能力。

While this provided a lot of test coverage, it didn’t actually test the reason the device was used, which was to read data from RTUs. A simulator was written in Ruby with a FOX (FXRuby) GUI. This allowed mock data to be fed into the device. Because the simulator could be controlled remotely, it was incorporated into automated tests that exercised the embedded device’s ability to read data, respond to error conditions, and generate alarms when the input data exceeded a predetermined threshold.

嵌入式测试技术性很强,但借助模拟器提供的强大功能,整个团队都能够参与到设备测试中。编写模拟器是为了支持测试团队的测试,但固件程序员发现它很有价值,并用它来帮助他的开发工作。这是一个积极的意外副作用。象限 2 支持团队的测试可能会结合各种技术,就像他们在这个项目中所做的那样。

Embedded testing is highly technical, but with the power provided by the simulator, the whole team was able to participate in testing the device. The simulator was written to support testing for the test team, but the programmer for the firmware found it valuable and used it to help with his development efforts as well. That was a positive unexpected side effect. Quadrant 2 tests that support the team may incorporate a variety of technologies, as they did in this project.

通过面向业务的测试来评价产品

Critiquing the Product with Business-Facing Tests

本节概述了评价产品的面向业务的测试。

The business-facing tests that critique the product are outlined in this section.

探索性测试

Exploratory Testing

自动化测试简单易用,团队中的每个人都可以使用。可以运行单独的测试脚本来设置特定条件,允许有效无需花费大量时间手动输入数据即可完成探索性测试。这适用于所有三个测试框架:功能、Web 服务和嵌入式。

The automated tests were simple and easy for everyone on the team to use. Individual test scripts could be run to set up specific conditions, allowing effective exploratory testing to be done without having to spend a lot of time manually entering data. This worked for all three test frameworks: functional, web services, and embedded.

探索性测试、可用性测试和其他象限 3 测试在第 10 章评价产品的面向业务的测试”中讨论。

Exploratory testing, usability testing, and other Quadrant 3 tests are discussed in Chapter 10, “Business-Facing Tests that Critique the Product.”

该团队进行了探索性测试以补充自动化测试套件并获得最佳覆盖率。这种与系统的人机交互发现了自动化没有发现的问题。

The team performed exploratory testing to supplement the automated test suites and get the best coverage possible. This human interaction with the system found issues that automation didn’t find.

可用性测试不是系统的关键要求,但测试人员会观察,以便界面有意义并且流畅。测试人员广泛使用探索性测试来评价产品。产品工程师还使用探索性测试来进行他的解决方案验证测试。

Usability testing was not a critical requirement for the system, but the testers watched so that the interface made sense and flowed smoothly. The testers used exploratory testing extensively to critique the product. The product engineer also used exploratory testing for his solution verification tests.

测试数据源

Testing Data Feeds

如图 12-2所示,来自系统的数据在 JMS 队列和 Web 浏览器上可用。为了测试 JMS 队列,开发组编写了一个 Java 代理。它连接到队列并将任何到达的数据打印到控制台。他们还编写了一个 Ruby 客户端,通过管道接收这些数据,然后在 Ruby 自动测试系统中可用。

As shown in Figure 12-2, the data from the system is available on a JMS queue, as well as the web browser. To test the JMS queue, the development group wrote a Java proxy. It connected to a queue and printed any arriving data to the console. They also wrote a Ruby client that received this data via a pipe, which was then available in the Ruby automated test system.

遇到警报情况时会自动发送电子邮件。警报电子邮件包含纯文本电子邮件和带附件的电子邮件。MIME附件包含对测试有用数据,因此编写了支持附件的 Ruby 电子邮件客户端。

Emails were automatically sent when alarm conditions were encountered. The alarm emails contained both plain text email and email with attachments. The MIME attachments contained data useful for testing, so a Ruby email client that supported attachments was written.

端到端测试

The End-to-End Tests

象限 3 包括端到端功能测试,用于演示系统每个部分的预期行为。从一开始就很明显,只有使用所有组件才能确定整个远程数据监控系统是否正确运行。一旦编写了模拟器、嵌入式设备测试、Web 服务测试和应用程序测试,将它们组合起来生成整个系统的自动化测试就相对简单了。再次使用 Excel 电子表格来保存测试数据,并编写 Ruby 类来访问数据和预期结果。

Quadrant 3 includes end-to-end functional testing that demonstrates the desired behavior of every part of the system. From the beginning, it was apparent that correct operation of the whole Remote Data Monitoring system could only be determined when all components were used. Once the simulator, embedded device tests, web services tests, and application tests were written, it was a relatively simple matter to combine them to produce an automated test of the entire system. Once again, Excel spreadsheets were used to hold the test data, and Ruby classes were written to access the data and expected results.

卫星传输路径的不可预测响应使端到端测试变得复杂。设置了预定义的超时值,如果测试的实际值与预期值不匹配,测试将循环直到匹配或达到超时。当超时到期时,测试被视为失败。发现并消除了大多数传输问题这边走。手动测试不太可能发现它们,因为它们是零星的问题。

The end-to-end tests were complicated by the unpredictable response of the satellite transmission path. A predefined timeout value was set, and if the test’s actual value did not match the expected value, the test would cycle until it matched or the timeout was reached. When the timeout expired, the test was deemed to have failed. Most transmission issues were found and eliminated this way. It would have been highly unlikely that they would have been found with manual testing, because they were sporadic issues.

因为诸如此类的端到端测试可能很脆弱,所以它们可能不会作为自动回归套件的一部分保留。如果系统的所有组件都很好地覆盖了自动回归测试,则可能不需要自动端到端测试。但是,由于该系统的性质,如果没有自动化就不可能进行完整的测试。

Because end-to-end tests such as these can be fragile, they may not be kept as part of the automated regression suite. If all of the components of the system are well covered with automated regression tests, automated end-to-end tests might not be necessary. However, due to the nature of this system, it wasn’t possible to do a full test without automation.

用户验收测试

User Acceptance Testing

用户验收测试 (UAT) 是客户对产品的最终评价,客户本应从一开始就参与到项目中。在此示例中,真正的客户在法国,距离开发团队数千英里。该团队必须具有创造力才能获得成功的 UAT。客户在这一年中与团队成员一起工作了几次,因此与他们从未见过面相比,他们能够更轻松地与团队互动。

User Acceptance Testing (UAT) is the final critique of the product by the customer, who should have been involved in the project from the start. In this example, the real customer was in France, thousands of miles from the development team. The team had to be inventive to have a successful UAT. The customer came to work with the team members a couple of times during the year and so was able to interact with the team a little easier than if they’d never met.

在团队引入敏捷开发后,Janet 前往法国,在客户现场促成第一个 UAT。它工作得相当好,在修复了一些关键问题后,该版本被接受了。团队从那次经历中学到了很多东西。

After the team introduced agile development, Janet went to France to facilitate the first UAT at the customer site. It worked fairly well, and the release was accepted after a few critical issues were fixed. The team learned a lot from that experience.

第二次 UAT 签核是在内部完成的。为了做好准备,该团队与客户合作开发了一组客户可以执行的测试来验证新功能。客户能够在整个开发周期中测试应用程序,因此 UAT 没有产生任何问题。客户来了,进行了测试,并在一天内签收了。

The second UAT sign-off was done in-house. To prepare, the team worked with the customer to develop a set of tests the customer could perform to verify new functionality. The customer was able to test the application throughout the development cycle, so UAT didn’t produce any issues. The customer came, ran through the tests, and signed off in a day.

我们怎么强调与客户合作的重要性都不为过。尽管产品工程师是客户的代理人,但与实际客户面对面交流至关重要。随着时间的推移建立的关系对于项目的成功至关重要。珍妮特坚信 UAT 的成功是因为客户知道团队一路走来所做的事情。

We cannot stress the importance of working with the customer enough. Even though the product engineer was the proxy for the customer, it was crucial to get face time with the actual customer. The relationship that had been built over time was critical to the success of the project. Janet strongly believes that the UAT succeeded because the customer knew what the team was doing along the way.

可靠性

Reliability

可靠性是第 4 象限测试解决的“问题”之一,它是系统的一个关键因素,因为它监视的是经常出现故障的远程站点人迹罕至,尤其是在冬天。为测试嵌入式系统而开发的模拟器被设置在一个单独的环境中,并且一次运行数周以测量整个系统的稳定性(又一个“ility”)。可以根据需要计划和编码对系统设计的更正。这是一个很好的例子,说明为什么您不应该等到项目结束才进行批评产品的面向技术的测试。

Reliability, one of the “ilities” addressed by Quadrant 4 tests, was a critical factor of the system because it was monitoring remote sites that were often inaccessible, especially in winter. The simulator that was developed for testing the embedded system was set up on a separate environment, and was run for weeks at a time measuring stability (yet another “ility”) of the whole system. Corrections to the system design could be planned and coded as needed. This is a good example of why you shouldn’t wait until the end of the project to do the technology-facing tests that critique the product.

请参阅第 10 章,“评估产品的面向业务的测试”,了解更多关于可靠性测试等象限 4 测试的信息。

See Chapter 10, “Business-Facing Tests that Critique the Product,” for more about Quadrant 4 tests such as reliability testing.

文档

Documentation

本节介绍了记录所采用的方法。

The approach taken to documentation is presented in this section.

记录测试代码

Documenting the Test Code

在开发过程中,很明显测试代码需要一个正式的文档系统。最简单的解决方案是使用 RDoc,类似于 Javadoc,但用于 Ruby。RDoc 从源代码中提取带标签的注释,并生成包含文件、类和方法详细信息的网页。这些文档是每晚使用批处理文件生成的,并且可供整个团队使用。很容易找到创建了哪些测试装置。

During development, it became clear that a formal documentation system was needed for the test code. The simplest solution was to use RDoc, similar to Javadoc, but for Ruby. RDoc extracted tagged comments from the source code and generated web pages with details of files, classes, and methods. The documents were generated every night using a batch file and were available to the complete team. It was easy to find what test fixtures were created.

测试代码的文档有助于记录测试,并使我们更容易找到我们正在测试的内容以及测试做了什么。它非常强大且易于使用。

The documentation of the test code helped to document the tests and make it easier to find what we were testing and what the tests did. It was very powerful and easy to use.

报告测试结果

Reporting the Test Results

尽管正在进行综合测试,但在测试团队之外几乎没有证据表明这一点。自动化测试期间生成的日志为跟踪问题提供了很好的信息,但不适合更广泛的受众。

Although comprehensive testing was being performed, there was little evidence of this outside of the test team. The logs generated during automated tests provided good information to track down problems but were not suitable for a wider audience.

第 16 章,“开始行动”,给出了团队报告测试结果的方式的更多示例。

Chapter 16, “Hit the Ground Running,” gives more examples of ways teams report test results.

为了提高正在执行的测试的可见性,测试团队使用 Apache、PHP 和 mySQL 开发了一个日志记录和报告系统。当测试运行时,它将结果记录到数据库中。Web 前端允许项目利益相关者确定运行了哪些测试、通过/失败率和其他信息。

To raise the visibility of the tests being performed, the test team developed a logging and reporting system using Apache, PHP, and mySQL. When a test ran, it logged the result into the database. A web front end allowed project stakeholders to determine what tests were run, the pass/failure rate, and other information.

第 18 章,“编码和测试”,还讨论了可视化大图表的使用。

Chapter 18, “Coding and Testing,” also discusses uses of big visible charts.

我们还相信尽可能让我们的进步(好或坏)可见。为此,我们在此过程中创建了图表和图形,并将它们张贴在公共区域。图 12-4显示了我们创建的一些图表。

We also believed in making our progress visible (good or bad) as much as possible. To this end we created charts and graphs along the way and posted them in common areas. Figure 12-4 shows some of the charts we created.

图12-4远程监控系统项目组使用的大可视化图表

Figure 12-4 Big visible charts used by the remote monitoring system project team

图像

使用敏捷测试象限

Using the Agile Testing Quadrants

此示例演示了如何在复杂开发项目的生命周期中结合来自所有四个敏捷测试象限的测试实践以实现成功交付。这个团队的经验说明了我们一直强调的许多原则。整个团队,包括程序员、测试人员、客户代理人和实际客户,都为解决自动化问题做出了贡献。他们尝试了不同的方法。他们以不同的方式结合了自己开发的和开源的工具来执行所有级别的测试,从单元级别到端到端系统测试和 UAT。项目的成功证明了测试方法的成功。

This example demonstrates how testing practices from all four agile testing quadrants are combined during the life of a complex development project to achieve successful delivery. The experience of this team illustrates many of the principles we have been emphasizing. The whole team, including programmers, testers, customer proxy, and the actual customer, contributed to efforts to solve automation problems. They experimented with different approaches. They combined their homegrown and open source tools in different ways to perform testing at all levels, from the unit level to end-to-end system testing and UAT. The success of the project demonstrates the success of the testing approach.

当您计划每个史诗、发布或迭代时,与您的客户团队合作以了解业务优先级并分析风险。使用象限来帮助确定将需要的所有不同类型的测试以及何时应该执行它们。性能是最重要的标准吗?是个最高优先级与其他系统接口的能力?可用性可能是最重要的方面吗?

As you plan each epic, release, or iteration, work with your customer team to understand the business priorities and analyze risks. Use the quadrants to help identify all of the different types of testing that will be needed and when they should be performed. Is performance the most important criteria? Is the highest priority the ability to interface with other systems? Is usability perhaps the most important aspect?

投资于适应被测系统复杂性的测试架构。计划在适当的时间获得必要的资源和专业知识以进行专门测试。对于每种类型的测试,您的团队应该共同努力选择能够解决您的测试问题的工具。使用回顾来持续评估您的团队是否拥有成功所需的资源,以及是否及时指定了所有必要的测试以实现其目的,并适当地自动化。

Invest in a test architecture that accommodates the complexity of the system under test. Plan to obtain necessary resources and expertise at the right time for specialized tests. For each type of test, your team should work together to choose tools that solve your testing problems. Use retrospectives to continually evaluate whether your team has the resources it needs to succeed, and whether all necessary tests are being specified in time to serve their purpose, and automated appropriately.

端到端测试似乎不可能做到吗?您的团队是否发现很难编写单元测试?正如 Janet 的团队所做的那样,让每个人都尝试不同的方法和工具。这些象限提供了一个框架,可以就创造性方法进行富有成效的头脑风暴,以实现让团队为业务创造价值的测试。

Does end-to-end testing seem impossible to do? Is your team finding it hard to write unit tests? As Janet’s team did, get everyone experimenting with different approaches and tools. The quadrants provide a framework for productive brainstorming on creative ways to achieve the testing that will let the team deliver value to the business.

概括

Summary

在本章中,我们描述了一个真实的项目,该项目使用来自所有四个敏捷测试象限的测试来克服困难的测试挑战。我们使用该项目中的示例来展示团队如何成功进行所有类型的测试。远程数据监控系统项目的一些重要经验教训是:

In this chapter, we described a real project that used tests from all four agile testing quadrants to overcome difficult testing challenges. We used examples from this project to show how teams can succeed with all types of testing. Some important lessons from the Remote Data Monitoring System project are:

图像整个团队应该选择或创建解决每个测试问题的工具。

The whole team should choose or create tools that solve each testing problem.

图像可能需要结合使用电子表格和自定义编写的测试脚本等常用业务工具来完成复杂的测试。

Combinations of common business tools such as spreadsheets and custom-written test scripts may be needed to accomplish complex tests.

图像花时间构建适用于所有团队成员的正确测试架构。

Invest time in building the right test architecture that works for all team members.

图像想方设法让客户参与所有类型的测试,即使他们身处偏远地区。

Find ways to keep customers involved in all types of testing, even if they’re in a remote location.

图像以一种让所有涉众都了解迭代和项目进度的方式报告测试结果。

Report test results in a way that keeps all stakeholders informed about the iteration and project progress.

图像不要忘记记录 . . . 但只有有用的东西。

Don’t forget to document . . . but only what is useful.

图像考虑整个开发周期中测试的所有四个象限。

Think about all four quadrants of testing throughout your development cycles.

图像使用在测试期间吸取的经验教训来评论产品,以推动后续迭代中的开发。

Use lessons learned during testing to critique the product in order to drive development in subsequent iterations.

第四部分自动化

Part IV Automation

测试自动化是核心的敏捷实践。敏捷项目依赖于自动化。足够好的自动化可以让团队腾出时间来频繁地交付高质量的代码。它提供了一个框架,让团队在保持高标准的同时最大限度地提高速度。源代码控制、自动化构建和测试套件、部署、监控以及各种脚本和工具消除了单调乏味,确保了可靠性,并让团队始终尽最大努力工作。

Test automation is a core agile practice. Agile projects depend on automation. Good-enough automation frees the team to deliver high-quality code frequently. It provides a framework that lets the team maximize its velocity while maintaining a high standard. Source code control, automated builds and test suites, deployment, monitoring, and a variety of scripts and tools eliminate tedium, ensure reliability, and allow the team to do its best work at all times.

自动化也是一个广泛的话题。它包括编写简单的 shell 脚本、设置会话属性和创建强大的自动化测试等任务。随着我们了解更好的软件开发方法,自动化工具的范围和数量似乎呈指数级增长。令人高兴的是,教授自动化方法的优秀书籍的数量似乎增长得同样快。

Automation is also a vast topic. It includes tasks like writing simple shell scripts, setting up session properties, and creating robust automated tests. The range and number of automated tools seem to grow exponentially as we learn about better ways to produce software. Happily, the number of excellent books that teach ways to automate appears to grow just as fast.

本书着重于测试人员在敏捷开发中的角色。因为自动化是敏捷开发成功的关键,所以我们需要讨论它,但我们不能开始涵盖该主题的每个方面。我们确实想解释的是为什么您作为测试人员必须拥抱自动化,以及您和您的团队如何克服可能阻碍您的自动化工作的许多障碍。本节介绍如何应用敏捷价值观、原则和实践来发展实用的自动化策略、克服障碍并推动测试自动化。

This book is focused on the tester’s role in agile development. Because automation is key to successful agile development, we need to talk about it, but we can’t begin to cover every aspect of the subject. What we do want to explain is why you, as a tester, must embrace automation, and how you and your team can overcome the many obstacles that can hamper your automation efforts. This section describes how you can apply agile values, principles, and practices to grow a practical automation strategy, overcome barriers, and get traction on test automation.

第 13 章 为什么我们想要自动化测试以及是什么阻碍了我们

Chapter 13 Why We Want to Automate Tests and What Holds Us Back

图像

为什么我们要自动化测试、构建过程、部署和其他任务?敏捷团队专注于始终拥有可工作的软件,这使他们能够根据需要经常发布生产就绪的软件。实现这个目标需要不断的测试。在本章中,我们将探讨我们想要实现自动化的原因以及难以实现自动化的挑战。

Why do we automate testing, the build process, deployment, and other tasks? Agile teams focus on always having working software, which enables them to release production-ready software as often as needed. Achieving this goal requires constant testing. In this chapter, we look at reasons we want to automate and the challenges that make it hard to get traction on automation.

为什么要自动化?

Why Automate?

除了我们所说的您需要自动化才能成功使用敏捷之外,还有多种原因可以实现自动化。我们的列表包括以下内容:

There are multiple reasons to automate besides our saying you need to have automation to be successful using agile. Our list includes the following:

让我们更详细地探讨其中的每一个。

Let’s explore each of these in a little more detail.

手动测试耗时过长

Manual Testing Takes Too Long

团队想要自动化的最基本原因是手动完成所有必要的测试需要太长时间。随着您的应用程序变得越来越大,测试所有内容的时间也越来越长,有时呈指数增长,具体取决于 AUT(被测应用程序)的复杂性。

The most basic reason a team wants to automate is that it simply takes too long to complete all of the necessary testing manually. As your application gets bigger and bigger, the time to test everything grows longer and longer, sometimes exponentially, depending on the complexity of the AUT (Application under test).

敏捷团队能够通过每天拥有生产就绪的软件,在每次短迭代结束时交付生产就绪的软件。至少每天运行一整套通过回归测试是必不可少的做法,而手动回归测试无法做到这一点。如果您现在没有任何自动化,您将不得不手动进行回归测试,但不要让它阻止您开始自动化。

Agile teams are able to deliver production-ready software at the end of each short iteration by having production-ready software every day. Running a full suite of passing regression tests at least daily is an indispensable practice, and you can’t do it with manual regression testing. If you don’t have any automation now, you’ll have to regression test manually, but don’t let that stop you from starting to automate it.

如果您手动执行回归测试,那么每天、每次迭代都会花费越来越多的时间进行测试。为了让测试跟上编码的步伐,要么程序员必须花时间帮助进行手动回归测试,要么团队必须雇用更多的测试人员。不可避免地,技术债务和挫败感都会增加。

If you execute your regression testing manually, it takes more and more time testing every day, every iteration. In order for testing to keep pace with coding, either the programmers have to take time to help with manual regression testing, or the team has to hire more testers. Inevitably, both technical debt and frustration will grow.

如果代码甚至不必通过一套自动化的单元级回归测试,测试人员可能会花费大量时间研究、尝试重现和报告那些简单的错误,而花更少的时间来发现潜在的严重系统级错误。此外,由于团队没有进行测试先行的开发,代码设计的可测试性更差,可能无法提供业务所需的功能。

If the code doesn’t even have to pass a suite of automated unit level regression tests, the testers will probably spend much of their time researching, trying to reproduce and report those simple bugs, and less time finding potentially serious system level bugs. In addition, because the team isn’t doing test-first development, code design is more likely to be less testable and may not provide the functionality desired by the business.

手动测试许多不同的场景可能会花费很多时间,尤其是当您将输入键入用户界面时。如果您没有自动化的方法来加速,那么为各种复杂场景设置数据可能是一项艰巨的任务。结果,可能只测试了有限数量的场景,并且可能会遗漏重要的缺陷。

Manually testing a number of different scenarios can take a lot of time, especially if you’re keying inputs into a user interface. Setting up data for a variety of complex scenarios can be an overwhelming task if you have no automated way to speed it up. As a result, only a limited number of scenarios may be tested, and important defects can be missed.

手动流程容易出错

Manual Processes Are Error Prone

手动测试变得重复,特别是如果您遵循脚本测试,并且手动测试很快就会变得无聊。犯错太容易了,即使是简单的错误也会被忽视。将跳过步骤甚至整个测试。如果团队面临紧迫的截止日期,就会有偷工减料的诱惑,结果就是错过了问题。

Manual testing gets repetitive, especially if you’re following scripted tests, and manual tests get boring very quickly. It’s way too easy to make mistakes and overlook even simple bugs. Steps and even entire tests will be skipped. If the team’s facing a tight deadline, there’s a temptation to cut corners, and the result is a missed problem.

由于手动测试很慢,您可能仍在迭代的最后一天的午夜进行测试。那么你会注意到多少错误?

Because manual testing is slow, you might still be testing at midnight on the last day of the iteration. How many bugs will you notice then?

自动化构建、部署、版本控制和监控也大大有助于降低风险并使您的开发过程更加一致。自动化这些脚本化测试消除了出错的可能性,因为每次测试都是以完全相同的方式完成的。

Automated builds, deployment, version control, and monitoring also go a long way toward mitigating risk and making your development process more consistent. Automating these scripted tests eliminate the possibility of errors, because each test is done exactly the same way every time.

“一次构建,多次部署”的格言是测试人员的梦想成真。构建和部署过程的自动化使您能够准确了解您在任何给定环境中测试的内容。

The adage of “build once, deploy to many” is a tester’s dream come true. Automation of the build and deploy processes allow you to know exactly what you are testing on any given environment.

自动化使人们能够尽最大努力工作

Automation Frees People to Do Their Best Work

编写测试先行的代码有助于程序员理解需求并相应地设计代码。让持续构建运行所有单元测试和功能回归测试意味着有更多时间进行有趣的探索性测试。自动化探索性测试的设置意味着有更多时间来探索系统的潜在薄弱部分。因为您没有花时间执行繁琐的手动脚本,所以您有精力做好工作,思考不同的场景并更多地了解应用程序的工作原理。

Writing code test-first helps programmers understand requirements and design code accordingly. Having continual builds run all of the unit tests and the functional regression tests means more time to do interesting exploratory testing. Automating the setup for exploratory tests means even more time to probe into potentially weak parts of the system. Because you didn’t spend time executing tedious manual scripts, you have the energy to do a good job, thinking of different scenarios and learning more about how the application works.

如果我们一直在思考如何对修复或新功能进行自动化测试,我们更有可能想到可测试性和质量设计,而不是可能证明脆弱的快速破解。这意味着更好的代码和更好的测试。

If we’re thinking constantly about how to automate tests for a fix or new feature, we’re more likely to think of testability and a quality design rather than a quick hack that might prove fragile. That means better code and better tests.

自动化测试实际上可以帮助实现整个应用程序的一致性。

Automating tests can actually help with consistency across the application.

珍妮特的故事

Janet’s Story

Jason(我的一位测试员同事)和我正在使用 Ruby 和 Watir 编写一些 GUI 自动化脚本,并为测试的按钮名称添加常量。我们很快意识到每个页面上的按钮的命名并不一致。我们能够非常快速地更改它们并解决这些一致性问题,并且有一种简单的方法来执行命名约定。

Jason (one of my fellow testers) and I were working on some GUI automation scripts using Ruby and Watir, and were adding constants for button names for the tests. We quickly realized that the buttons on each page were not consistently named. We were able to get them changed and resolved those consistency issues very quickly, and had an easy way to enforce the naming conventions.

——珍妮特

—Janet

Pragmatic Project Automation [2004]等书籍可以指导您自动执行日常开发工作,让您的团队腾出时间进行重要活动,例如探索性测试。

Books such as Pragmatic Project Automation [2004] can guide you in automating daily development chores and free your team for important activities such as exploratory testing.

当好人可以自由地做最好的工作时,项目就会成功。适当地自动化测试可以实现这一点。检测现有功能变化并提供即时反馈的自动回归测试是其中的主要组成部分。

Projects succeed when good people are free to do their best work. Automating tests appropriately makes that happen. Automated regression tests that detect changes to existing functionality and provide immediate feedback are a primary component of this.

自动回归测试提供安全网

Automated Regression Tests Provide a Safety Net

大多数在软件行业工作了几年的从业者都知道,当他们面临修复错误或在设计不佳的代码中实现自动化测试未涵盖的新功能时的恐惧感。在这里挤压气球的一端,另一部分就会凸出。会坏吗?

Most practitioners who’ve been in the software business for a few years know the feeling of dread when they’re faced with fixing a bug or implementing a new feature in poorly designed code that isn’t covered by automated tests. Squeeze one end of the balloon here and another part of it bulges out. Will it break?

通过自动回归测试知道代码具有足够的覆盖率,给人以极大的信心。当然,更改可能会产生意想不到的效果,但如果是在单元级别,我们会在几分钟内知道,如果是在更高的功能级别,我们会在几小时内知道。让更改测试先行意味着在编写代码之前仔细考虑更改的行为并编写测试来验证它,这增加了这种信心。

Knowing the code has sufficient coverage by automated regression tests gives a great feeling of confidence. Sure, a change might produce an unexpected effect, but we’ll know about it within a matter of minutes if it’s at the unit level, or hours if at a higher functional level. Making the change test-first means thinking through the changed behavior before writing the code and writing a test to verify it, which adds to that confidence.

珍妮特的故事

Janet’s Story

我最近与我团队中的一位测试人员进行了交谈,他质疑自动化测试的价值。我的第一个回答是“这是团队的安全网”。然而,他对这个前提提出了质疑。难道我们只是变得依赖测试而不是解决问题的根本原因吗?

I recently had a conversation with one of the testers on my team who questioned the value of automated tests. My first answer was “It’s a safety net” for the team. However, he challenged that premise. Don’t we just become reliant on the tests rather than fixing the root cause of the problem?

这让我对我的回答有了更多的思考。在某种意义上他是对的;如果我们对我们的测试挑战沾沾自喜,仅仅依靠自动化测试来发现我们的问题,然后修复它们足以让测试通过,那么我们就是在伤害自己。

It made me think a bit more about my answer. He was right in one sense; if we become complacent about our testing challenges and depend solely on automated tests to find our issues, and then just fix them enough for the test to pass, we do ourselves a disservice.

但是,如果我们使用测试来识别问题区域并以正确的方式修复它们或根据需要进行重构,那么我们就是在以正确的方式使用自动化安全网。自动化对于敏捷项目的成功至关重要,尤其是随着应用程序规模的增长。

However, if we use the tests to identify problem areas and fix them the right way or refactor as needed, then we are using the safety net of automation in the right way. Automation is critical to the success of an agile project, especially as the application grows in size.

——珍妮特

—Janet

当他们没有作为安全网的自动化测试套件时,程序员可能会开始将测试人员本身视为安全网。很容易想象 Joe Programmer 的思考过程是这样的:“我应该返回并为 formatEmployeeInfo 添加一些自动化单元测试,但我知道 Susie Tester 将检查每个手动使用它的页面。她会看看是否有任何问题,所以我只是在重复她的努力。”

When they don’t have an automated suite of tests acting as a safety net, the programmers may start viewing the testers themselves as a safety net. It’s easy to imagine that Joe Programmer’s thought process goes like this: “I ought to go back and add some automated unit tests for formatEmployeeInfo, but I know Susie Tester is going to check every page where it’s used manually. She’ll see if anything is off, so I’d just be duplicating her effort.”

程序员对测试人员的才能评价如此之高是件好事,但 Joe 正在走下坡路。如果他不自动化这些单元测试,他可能会跳过哪些其他测试?Susie 将非常忙于浏览所有这些页面。

It’s nice that a programmer would think so highly of the tester’s talents, but Joe is headed down a slippery slope. If he doesn’t automate these unit tests, which other tests might he skip? Susie is going to be awfully busy eyeballing all those pages.

自动化回归测试覆盖率高的团队可以无所畏惧地更改代码。他们不必怀疑,“如果我更改这个 formatEmployeeInfo 模块,我会破坏用户界面中的某些东西吗?” 测试会立即告诉他们是否损坏了任何东西。他们可以比完全依赖手动测试的团队快得多。

Teams that have good coverage from automated regression tests can make changes to the code fearlessly. They don’t have to wonder, “If I change this formatEmployeeInfo module, will I break something in the user interface?” The tests will tell them right away whether or not they broke anything. They can go lots faster than teams relying exclusively on manual testing.

自动化测试会尽早且经常提供反馈

Automated Tests Give Feedback, Early and Often

一项功能的自动化测试通过后,它必须继续通过,直到功能被有意更改。当我们计划对应用程序进行更改时,我们会更改测试以适应它们。当自动化测试意外失败时,代码更改可能会引入回归缺陷。每次签入新代码时运行一套自动化测试有助于确保快速发现回归错误。快速反馈意味着更改在一些程序员的脑海中仍然是新鲜的,因此与直到几周后的某个测试阶段才发现错误相比,故障排除会更快。快速失败意味着修复错误的成本更低。

After an automated test for a piece of functionality passes, it must continue to pass until the functionality is intentionally changed. When we plan changes in the application, we change the tests to accommodate them. When an automated test fails unexpectedly, a regression defect may have been introduced by a code change. Running an automated suite of tests every time new code is checked in helps ensure that regression bugs will be caught quickly. Quick feedback means the change is still fresh in some programmer’s mind, so troubleshooting will go more quickly than if the bug weren’t found until some testing phase weeks later. Failing fast means bugs are cheaper to fix.

自动化测试定期运行,通常充当您的变更检测器。它们让团队有机会了解自上次构建以来发生了什么变化。例如,上一次构建是否有任何负面影响?如果您的自动化套件具有足够的覆盖范围,它可以很容易地分辨出手动测试人员永远无法找到的深远影响。

Automated tests run regularly and often act as your change detector. They allow the team an opportunity to know what has changed since the last build. For example, were there any negative side effects with the last build? If your automation suite has sufficient coverage, it can easily tell far-reaching effects that manual testers can never hope to find.

通常情况下,如果回归测试不是自动化的,它们就不会在每次迭代中运行,更不用说每天运行了。问题在比赛结束时很快出现,此时团队需要完成所有回归测试。本应及早发现的错误在游戏后期才被发现。早期测试的许多好处都失去了。

More often than not, if regression tests are not automated, they won’t get run every iteration, let alone every day. The problem arises very quickly during the end game, when the team needs to complete all of the regression tests. Bugs that would have been caught early are found late in the game. Many of the benefits of testing early are lost.

驱动编码可以做更多的测试和示例

Tests and Examples that Drive Coding Can Do More

第 7 章支持团队的面向技术的测试”中,我们讨论了使用测试和示例来驱动编码。我们已经讨论了通过单元测试和客户测试来驱动编码的重要性。我们还想强调,如果这些测试是自动化的,它们会因为不同的原因而变得有价值。它们成为非常强大的回归套件的基础。

In Chapter 7, “Technology-Facing Tests that Support the Team,” we talked about using tests and examples to drive coding. We’ve talked about how important it is to drive coding with both unit and customer tests. We also want to stress that if these tests are automated, they become valuable for a different reason. They become the base for a very strong regression suite.

丽莎的故事

Lisa’s Story

在我的团队掌握了单元测试、重构、持续集成和其他面向技术的实践之后,我们能够在开发过程中发现回归错误和错误实现的功能。

After my team got a handle on unit tests, refactoring, continuous integration, and other technology-facing practices, we were able to catch regression bugs and incorrectly implemented functionality during development.

当然,这并不意味着我们的问题就完全解决了;我们有时仍然会错过或误解要求。然而,拥有一个适当的自动化框架使我们能够开始专注于在前期测试中更好地捕获需求。我们也有更多时间进行探索性测试。随着时间的推移,我们的缺陷率急剧下降,而我们的客户对交付的商业价值的满意度却上升了。

Of course, this didn’t mean our problems were completely solved; we still sometimes missed or misunderstood requirements. However, having an automation framework in place enabled us to start focusing on doing a better job of capturing requirements in up-front tests. We also had more time for exploratory testing. Over time, our defect rate declined dramatically, while our customers’ delight in the delivered business value went up.

——丽莎

—Lisa

TDD 和 SDD(故事测试驱动开发)让团队始终以测试为先。在计划会议期间,他们会讨论测试和进行测试的最佳方法。他们设计代码使测试通过,因此可测试性从来都不是问题。自动化测试套件随着代码库一起增长,为持续重构提供安全网。整个团队实践 TDD 并始终如一地编写单元测试很重要,否则安全网就会出现漏洞。

TDD and SDD (story test-driven development) keep teams thinking test-first. During planning meetings, they talk about the tests and the best way to do them. They design code to make the tests pass, so testability is never an issue. The automated test suite grows along with the code base, providing a safety net for constant refactoring. It’s important that the whole team practices TDD and consistently writes unit tests, or the safety net will have holes.

参考书目包含 Jennitta Andrea [2008] 的一篇关于 TDD 团队礼仪的文章。

The bibliography contains an article by Jennitta Andrea [2008] on team etiquette for TDD.

团队也不会积累太多技术债务,而且他们的速度必然会稳定甚至随着时间的推移而增加。这就是为什么业务经理应该乐于让软件团队花时间正确实施良好实践的原因之一。

The team also doesn’t accrue too much technical debt, and their velocity is bound to be stable or even increase over time. That’s one of the reasons why the business managers should be happy to let software teams take the time to implement good practices correctly.

测试是很棒的文档

Tests Are Great Documentation

第 III 部分中,我们解释了敏捷团队如何使用示例和测试来指导开发。当说明所需行为示例的测试自动化时,它们将成为系统实际工作方式的“活”文档。最好有关于某项功能如何工作的叙述性文档,但没有人会反对用红色和绿色显示代码如何在给定输入集上运行的可执行测试。

In Part III, we explained how agile teams use examples and tests to guide development. When tests that illustrate examples of desired behavior are automated, they become “living” documentation of how the system actually works. It’s good to have narrative documentation about how a piece of functionality works, but nobody can argue with an executable test that shows in red and green how the code operates on a given set of inputs.

静态文档很难保持最新,但如果我们在系统发生变化时不更新我们的自动化测试,测试就会失败。我们需要修复它们以保持我们的构建过程“绿色”。这意味着自动化测试始终是我们代码工作方式的准确描述。这只是我们在自动化方面的投资获得回报的方式之一。

It’s hard to keep static documentation up to date, but if we don’t update our automated tests when the system changes, the tests fail. We need to fix them to keep our build process “green.” This means that automated tests are always an accurate picture of how our code works. That’s just one of the ways our investment in automation pays off.

投资回报率和回报

ROI and Payback

刚才提出的所有原因都有助于自动化的底线和回报。自动化为项目提供了一致性,并为团队提供了以不同方式进行测试并突破应用程序极限的机会。自动化意味着测试人员和团队成员有更多时间专注于及时将正确的产品推向市场。

All of the reasons just presented contribute to the bottom line and the payback of automation. Automation provides consistency to a project and gives the team opportunity to test differently and push the limits of the application. Automation means extra time for testers and team members to concentrate on getting the right product out to market in a timely manner.

测试自动化回报的一个重要组成部分是修复缺陷的方式。依赖手动测试的团队往往在编写包含错误的代码很久之后才发现错误。他们进入修复“当天的错误”的模式,而不是查看错误的根本原因并相应地重新设计代码。当程序员在他们自己的沙盒中运行自动化测试套件时,自动化回归测试会在代码签入之前发现错误,因此有时间纠正设计。这是一个更大的回报,也是你减少技术债务和开发可靠代码的方式。

An important component of test automation payback is the way defects are fixed. Teams that rely on manual tests tend to find bugs long after the code containing the bug is written. They get into the mode of fixing the “bug of the day,” instead of looking at the root cause of the bug and redesigning the code accordingly. When programmers run the automated test suite in their own sandbox, the automated regression tests find the bugs before the code is checked in, so there’s time to correct the design. That’s a much bigger payback, and it’s how you reduce technical debt and develop solid code.

自动化的障碍——阻碍的事情

Barriers to Automation—Things that Get in the Way

早在 2001 年,Bret Pettichord [ 2001 ] 就列出了困扰自动化的七个问题。它们仍然适用,但适用于未将自动化作为其开发一部分的团队。当然,因为你在做敏捷,所以你在做,对吧?

Back in 2001, Bret Pettichord [2001] listed seven problems that plague automation. They are still applicable, but are intended for teams that do not incorporate automation as part of their development. And of course, because you are doing agile, you are doing that, right?

我们认为每个人都将自动化任务作为每个故事的一部分,但事实是,如果一切都在您的控制之下,您可能不会阅读本节。我们已经包含了 Bret 的列表,以显示如果您不将自动化作为日常项目可交付成果的一部分,您可能遇到的问题。

We would like to think that everyone has included automation tasks as part of each story, but the reality is that you probably wouldn’t be reading this section if you had it all under control. We’ve included Bret’s list to show what problems you probably have if you don’t include automation as part of the everyday project deliverables.

布雷特的名单

Bret’s List

Bret 的自动化问题列表如下所示:

Bret’s list of automation problems looks like this:

图像仅将业余时间用于测试自动化并不能为其提供所需的重点。

Only using spare time for test automation doesn’t give it the focus it needs.

图像缺乏明确的目标。

There is a lack of clear goals.

图像缺乏经验。

There is a lack of experience.

图像人员流动率很高,因为你失去了你可能拥有的任何经验。

There is high turnover, because you lose any experience you may have.

图像对绝望的反应通常是选择自动化的原因,在这种情况下,它可能更像是一个愿望,而不是一个现实的建议。

A reaction to desperation is often the reason why automation is chosen, in which case it can be more of a wish than a realistic proposal.

图像 可能不愿意考虑测试;乐趣在于自动化,而不是测试。

There can be a reluctance to think about testing; the fun is in the automating, not in the testing.

图像专注于解决技术问题可能会使您忽视结果是否满足测试需要。

Focusing on solving the technology problem can cause you to lose sight of whether the result meets the testing need.

我们认为团队在尝试自动化时还会遇到其他一些问题。即使我们确实尝试在我们的项目可交付成果中包含自动化,也存在其他阻碍成功的障碍。在下一节中,我们列出了成功测试自动化的障碍。

We think there are some other problems that teams run into when trying to automate. Even if we do try to include automation in our project deliverables, there are other barriers to success. In the next section, we present our list of obstacles to successful test automation.

我们的名单

Our List

我们列出的成功测试自动化的障碍是基于我们自己的敏捷团队以及我们认识的其他团队的经验。

Our list of barriers to successful test automation is based on the experiences we’ve had with our own agile teams as well as that of the other teams we know.

程序员的态度——“为什么要自动化?”

Programmers’ Attitude—“Why Automate?”

习惯于在传统环境中工作的程序员,在传统环境中,一些独立的、看不见的 QA 团队进行所有测试,甚至可能不会对功能测试自动化进行太多思考。有些程序员懒得做太多测试,因为他们有 QA 团队作为安全网,可以在发布前发现错误。长的瀑布式开发周期使得测试对程序员来说更加遥远。当看不见的测试人员开始工作时,程序员已经转移到下一个版本。缺陷排在队列中,以后要花费巨大的代价来修复,而且没有人对产生它们负责。即便是采用了测试驱动开发,习惯于在单元级别进行自动化测试的程序员,也可能不会去思考超出单元级别的验收测试是如何完成的。

Programmers who are used to working in a traditional environment, where some separate, unseen QA team does all of the testing, may not even give functional test automation a lot of thought. Some programmers don’t bother to test much because they have the QA team as a safety net to catch bugs before release. Long waterfall development cycles make testing even more remote to programmers. By the time the unseen testers are doing their job, the programmers have moved on to the next release. Defects go into a queue to be fixed later at great expense, and nobody is accountable for having produced them. Even programmers who have adopted test-driven development and are used to automating tests at the unit level may not think about how acceptance tests beyond the unit level get done.

丽莎的故事

Lisa’s Story

我曾经加入了一个 XP 团队,该团队由熟练的程序员组成,他们从事测试驱动开发,该团队在自动构建过程中运行了一套合理的单元测试。他们从未自动化过任何面向业务的测试,所以有一天我开始讨论他们可能使用什么工具来自动化面向业务的功能性回归测试。程序员想知道为什么我们需要自动化这些测试。

I once joined an XP team of skilled programmers practicing test-driven development that had a reasonable suite of unit tests running in an automated build process. They had never automated any business-facing tests, so one day I started a discussion about what tools they might use to automate functional business-facing regression tests. The programmers wanted to know why we needed to automate these tests.

在第一次迭代结束时,当每个人都在手工执行验收测试时,我指出除了所有新故事的测试之外,所有这些测试都将在下一次迭代中作为回归测试再次进行. 在第三次迭代中,测试次数将增加三倍。对于测试人员来说,这似乎显而易见,但有时程序员需要先进行手动测试,然后才能理解自动化测试的必要性。

At the end of the first iteration, when everyone was executing the acceptance tests by hand, I pointed out that there would be all these tests to do again in the next iteration as regression tests, in addition to the tests for all of the new stories. In the third iteration, there would be three times as many tests. To a tester, it seems ridiculously obvious, but sometimes programmers need to do the manual tests before they understand the compulsion to automate them.

——丽莎

—Lisa

教育是让程序员和团队其他成员了解自动化重要性的关键。

Education is the key to getting programmers and the rest of the team to understand the importance of automation.

“痛苦的驼峰”(学习曲线)

The “Hump of Pain” (The Learning Curve)

学习测试自动化是很困难的,尤其是学习如何以一种对投入其中的资源产生良好回报的方式进行测试。我们听 Brian Marick 用来描述开发人员(包括测试人员)必须克服的自动化初始阶段的一个术语是“痛苦的驼峰”(见图 13-1 。这个短语指的是大多数团队在采用自动化时所经历的挣扎。

It’s hard to learn test automation, especially to learn how to do it in a way that produces a good return on the resources invested in it. A term we’ve heard Brian Marick use to describe the initial phase of automation that developers (including testers) have to overcome is the “hump of pain” (see Figure 13-1). This phrase refers to the struggle that most teams go through when adopting automation.

图 13-1自动化学习曲线的痛峰

Figure 13-1 Hump of pain of the automation learning curve

图像

通常期望新团队采用 TDD 和重构等难以学习的实践。如果没有良好的指导、充足的时间来掌握新技能以及强大的管理支持,他们很容易灰心丧气。如果他们在学习上有额外的障碍,例如必须使用设计不佳的遗留代码,那么似乎不可能在测试自动化上获得牵引力。

New teams are often expected to adopt practices such as TDD and refactoring, which are difficult to learn. Without good coaching, plenty of time to master new skills, and strong management support, they’re easily discouraged. If they have extra obstacles to learning, such as having to work with poorly designed legacy code, it may seem impossible to ever get traction on test automation.

丽莎的故事

Lisa’s Story

我在 ePlan Services 的团队最初尝试为一个遗留系统编写单元测试,但在编写时绝对没有考虑到测试。他们发现这是一项艰巨的任务,如果不是不可能的话,所以他们决定在一个新的、可测试的架构中编写所有新故事。有趣的是,大约一年后,他们发现为旧代码编写单元测试并没有那么难。问题是他们根本不知道如何编写单元测试,而且在设计良好的架构上更容易学习。编写单元级测试成为编写代码的自然组成部分。

My team at ePlan Services originally tried to write unit tests for a legacy system that definitely wasn’t written with testing in mind. They found this to be a difficult, if not impossible, task, so they decided to code all new stories in a new, testable architecture. Interestingly, about a year later, they discovered it wasn’t really that hard to write unit tests for the old code. The problem was they didn’t know how to write unit tests at all, and it was easier to learn on a well-designed architecture. Writing unit-level tests became simply a natural part of writing code.

——丽莎

—Lisa

由于您正在构建特定领域的测试框架或学习新的功能测试工具,因此可能会出现痛苦的驼峰。您可能需要聘请专家来帮助您正确设置。

The hump of pain may occur because you are building your domain-specific testing framework or learning your new functional test tool. You may want to bring in an expert to help you get it set up right.

你知道你的团队已经克服了当自动化变得(如果不容易)至少是一个自然而根深蒂固的过程时的“驼峰”。Lisa 曾在三个成功采用 TDD 和功能测试自动化的团队工作过。每一次,团队都需要大量的时间、培训、承诺和鼓励来推动实践。

You know your team has overcome the “hump” when automation becomes, if not easy, at least a natural and ingrained process. Lisa has worked on three teams that successfully adopted TDD and functional test automation. Each time, the team needed lots of time, training, commitment, and encouragement to get traction on the practices.

初始投资

Initial Investment

即使整个团队都在努力解决这个问题,自动化也需要大量投资,而且可能不会立即得到回报。决定使用什么测试框架以及是在内部构建它们还是使用外部生产的工具需要时间和研究。可能需要新的硬件和软件。团队成员可能需要一段时间才能熟悉如何使用自动化测试工具。

Even with the whole team working on the problem, automation requires a big investment, one that may not pay off right away. It takes time and research to decide on what test frameworks to use and whether to build them in-house or use externally produced tools. New hardware and software are probably required. Team members may take a while to ramp up on how to use automated test harnesses.

许多人都经历过没有回报的测试自动化努力。他们的组织可能已经购买了供应商捕获回放工具,并将其提供给 QA 团队,并期望它能够解决所有自动化问题。这些工具通常放在架子上积尘。可能已经生成了数千行 GUI 测试脚本,没有人知道它们在做什么,或者无法维护的测试脚本不再有用。

Many people have experienced test automation efforts that didn’t pay off. Their organization may have purchased a vendor capture-playback tool, given it to the QA team, and expected it to solve all of the automation problems. Such tools often sit on a shelf gathering dust. There may have been thousands of lines of GUI test scripts generated, with no one left who knows what they do, or the test scripts that are impossible to maintain are no longer useful.

珍妮特的故事

Janet’s Story

我作为新的 QA 经理走进了一个组织。我的任务之一是评估当前的自动化测试脚本并增加测试覆盖率。几年前购买了一个供应商工具,开发初始套件的测试人员已不在该组织。一名新聘用的测试人员正在尝试学习该工具并向套件中添加测试。

I walked into an organization as a new QA manager. One of my tasks was to evaluate the current automated test scripts and increase the test coverage. A vendor tool had been purchased a few years earlier, and the testers who had developed the initial suite were no longer with the organization. One of the new testers hired was trying to learn the tool and was adding tests to the suite.

我做的第一件事是让这个测试人员对测试套件进行评估,看看实际覆盖率是多少。她花了一周时间试图了解测试是如何组织的。我也开始四处寻找,发现现有的测试设计得很差,价值也很小。

The first thing I did was ask this tester to do an assessment on the test suite to see what the coverage actually was. She spent a week just trying to understand how the tests were organized. I started poking around as well and found that that the existing tests were very poorly designed and had very little value.

我们停止添加更多测试,而是花一点时间了解我们的测试自动化的目标是什么。事实证明,供应商工具无法完成我们真正需要它做的事情,因此我们取消了许可证并找到了一个满足我们需求的开源工具。

We stopped adding more tests and instead spent a little bit of time understanding what the goal was for our test automation. As it turned out, the vendor tool could not do what we really needed it to do, so we cancelled the licenses and found an open source tool that met our needs.

我们仍然需要花时间学习新的开源工具,但如果我们继续使用原始的供应商工具,那笔投资就已经完成了,因为团队中没有人知道如何使用原始工具。

We still had to spend time learning the new open source tool, but that investment would have been made if we’d stayed with the original vendor tool anyhow, because no one on the team knew how to use the original tool.

——珍妮特

—Janet

测试设计技能对自动化是否立即获得回报有着巨大的影响。糟糕的做法会产生难以理解和维护的测试,并可能产生难以解释的结果或需要时间研究的错误失败。培训和技能不足的团队可能会认为他们的自动化投资回报不值得他们花时间。

Test design skills have a huge impact on whether automation pays off right away. Poor practices produce tests that are hard to understand and maintain, and may produce hard-to-interpret results or false failures that take time to research. Teams with inadequate training and skills might decide the return on their automation investment isn’t worth their time.

良好的测试设计实践产生简单、设计良好、不断重构、可维护的测试。测试模块和对象的库会随着时间的推移而建立起来,并使新测试的自动化速度更快。有关自动化测试设计的一些提示和指南,请参阅第 14 章。

Good test design practices produce simple, well-designed, continually refactored, maintainable tests. Libraries of test modules and objects build up over time and make automating new tests quicker. See Chapter 14 for some hints on and guidelines for test design for automation.

我们知道捕获指标并不容易。例如,试图捕获编写和维护自动化测试所花费的时间与手动运行相同回归测试所花费的时间几乎是不可能的。同样,试图捕捉引入缺陷后几分钟内修复缺陷的成本与迭代结束后发现和修复问题的成本也相当困难。许多团队不努力跟踪这些信息。如果没有数字表明自动化需要更少的努力并提供更多的价值,团队就更难说服管理层相信对自动化的投资是值得的。缺乏证明自动化投资回报的指标也使得改变团队的习惯变得更加困难。

We know it’s not easy to capture metrics. For example, trying to capture the time it takes to write and maintain automated tests versus the time it takes to run the same regression tests manually is almost impossible. Similarly, trying to capture how much it costs to fix defects within minutes of introducing them versus how much it costs to find and fix problems after the end of the iteration is also quite difficult. Many teams don’t make the effort to track this information. Without numbers showing that automating requires less effort and provides more value, it’s harder for teams to convince management that an investment in automation is worthwhile. A lack of metrics that demonstrate automation’s return on investment also makes it harder to change a team’s habits.

始终在变化中的代码

Code that’s Always in Flux

通过用户界面进行自动化测试很棘手,因为 UI 在开发过程中往往会频繁更改。这就是简单的记录和回放技术很少成为敏捷项目好的选择的原因之一。

Automating tests through the user interface is tricky, because UIs tend to change frequently during development. That’s one reason that simple record and playback techniques are rarely a good choice for an agile project.

如果团队努力在底层业务逻辑和数据库访问方面做出良好的设计,并且经常进行重大返工,那么即使在 API 级别的 GUI 后面进行自动化测试也可能很难跟上。如果在设计系统时很少考虑测试,那么找到一种自动化测试的方法可能会很困难而且成本很高。程序员和测试人员需要一起工作以获得可测试的应用程序。

If the team is struggling to produce a good design on the underlying business logic and database access, and major rework is done frequently, it might be hard to keep up even with tests automated behind the GUI at the API level. If little thought is given to testing while designing the system, it might be difficult and expensive to find a way to automate tests. The programmers and testers need to work together to get a testable application.

尽管实际的代码和实现,如 GUI,在敏捷开发中往往会经常变化,但代码的意图很少改变。根据应用程序的意图而不是其实现来组织测试代码,使您能够跟上开发的步伐。

Although the actual code and implementation, like the GUI, tends to change frequently in agile development, the intent of code rarely changes. Organizing test code by the application’s intent, rather than by its implementation, allows you to keep up with development.

第 14 章敏捷测试自动化策略”中,我们将研究组织自动化测试的方法。

In Chapter 14, “An Agile Test Automation Strategy,” we’ll look at ways to organize automated tests.

遗留代码

Legacy Code

根据我们的经验,如果您在以测试为设计理念的架构中编写全新代码,则更容易获得自动化的关注。为很少或没有测试的现有代码编写测试充其量是一项艰巨的任务。对于刚接触敏捷和测试自动化的团队来说,这似乎几乎是不可能的。

In our experience, it’s much easier to get traction on automation if you’re writing brand new code in an architecture designed with testing in mind. Writing tests for existing code that has few or no tests is a daunting task at best. It seems virtually impossible to a team new to agile and new to test automation.

它有时是第 22 条军规。你想自动化测试,以便重构一些遗留代码,但遗留代码并不是为可测试性而设计的,所以即使在单元级别也很难自动化测试。

It is sometimes a Catch-22. You want to automate tests so you can refactor some of the legacy code, but the legacy code isn’t designed for testability, so it is hard to automate tests even at the unit level.

如果您的团队面临此类挑战并且没有预留足够的时间来集思广益如何解决它,那么就很难有效地开始自动化测试。第 14 章给出了解决这些问题的策略。

If your team faces this type of challenge and doesn’t budget plenty of time to brainstorm about how to tackle it, it’ll be tough to start automating tests effectively. Chapter 14 gives strategies to address these issues.

害怕

Fear

测试自动化对于那些从未掌握它的人来说是可怕的,甚至对于那些已经掌握它的人来说也是如此。程序员可能擅长编写生产代码,但他们可能在编写自动化测试方面经验不足。测试人员可能没有很强的编程背景,并且他们不相信自己潜在的测试自动化技能。

Test automation is scary to those who’ve never mastered it, and even to some who have. Programmers may be good at writing production code, but they might not be very experienced at writing automated tests. Testers may not have a strong programming background, and they don’t trust their potential test automation skills.

非编程测试人员经常得到的信息是,他们在敏捷世界中无能为力。我们不这么认为。没有个人测试员应该需要担心如何做自动化。这是一个团队问题,团队中通常有很多程序员可以提供帮助。诀窍是拥抱学习新想法。一次服用一天。

Non-programming testers have often gotten the message that they have nothing to offer in the agile world. We believe otherwise. No individual tester should need to worry about how to do automation. It’s a team problem, and there are usually plenty of programmers on the team who can help. The trick is to embrace learning new ideas. Take one day at a time.

旧习惯

Old Habits

当迭代进行得不顺利并且团队无法在迭代结束时完成所有的编程和测试任务时,团队成员可能会感到恐慌。我们观察到,当人们进入恐慌模式时,他们会养成舒适的旧习惯,即使这些习惯从未产生好的结果。

When iterations don’t proceed smoothly and the team can’t complete all of the programming and testing tasks by the end of an iteration, team members may panic. We’ve observed that when people go into panic mode, they fall into comfortable old habits, even if those habits never produced good results.

所以我们可能会说,“我们应该在 2 月 1 日交付。如果我们想赶上那个日期,我们就没有时间自动化任何测试了。” 我们将不得不在这段时间内完成任何可以完成的手动测试,并希望取得最好的成绩。我们以后总是可以自动化测试。”

So we may say, “We are supposed to deliver on February 1. If we want to meet that date, we don’t have time to automate any tests. We’ll have to do whatever manual tests can be done in that amount of time and hope for the best. We can always automate the tests later.”

这是灭亡之路。一些手动测试可以完成,但可能无法完成重要的手动探索性测试,这些测试可能会发现导致公司损失数十万美元销售损失的错误。然后,因为我们没有完成我们的测试自动化任务,这些任务会延续到下一个迭代,减少我们可以交付的业务价值。随着迭代的进行,情况继续恶化。

This is the road to perdition. Some manual tests can get done, but maybe not the important manual exploratory tests that would have found the bug that cost the company hundreds of thousands of dollars in lost sales. Then, because we didn’t finish with our test automation tasks, those tasks carry over to the next iteration, reducing the amount of business value we can deliver. As iterations proceed, the situation continues to deteriorate.

我们能克服这些障碍吗?

Can We Overcome These Barriers?

敏捷的全团队方法是克服自动化挑战的基础。刚接触敏捷的程序员可能习惯于交付代码而获得奖励,无论代码是否有错误,只要他们在截止日期前完成即可。测试驱动开发更多地面向设计而不是测试,因此面向业务的测试可能仍然没有进入他们的意识。它需要领导力和团队对质量的承诺,让每个人都思考如何编写、使用和运行面向技术和面向业务的测试。让整个团队参与测试自动化可能是一种文化挑战。

The agile whole-team approach is the foundation to overcoming automation challenges. Programmers who are new to agile are probably used to being rewarded for delivering code, whether it’s buggy or not, as long as they meet deadlines. Test-driven development is oriented more toward design than testing, so business-facing tests may still not enter their consciousness. It takes leadership and a team commitment to quality to get everyone thinking about how to write, use, and run both technology-facing and business-facing tests. Getting the whole team involved in test automation may be a cultural challenge.

请参阅第 3 章文化挑战”,了解有关改变团队文化以促进敏捷实践的一些想法。

See Chapter 3, “Cultural Challenges,” for some ideas on making changes to the team culture in order to facilitate agile practices.

在下一章中,我们将展示如何使用敏捷价值观和原则来克服我们在本章中描述的一些问题。

In the next chapter, we show how to use agile values and principles to overcome some of the problems we’ve described in this chapter.

概括

Summary

在本章中,我们分析了与测试自动化相关的一些重要因素:

In this chapter, we analyzed some important factors related to test automation:

图像我们需要自动化来提供安全网,为我们提供必要的反馈,将技术债务降至最低,并帮助推动编码。

We need automation to provide a safety net, provide us with essential feedback, keep technical debt to a minimum, and help drive coding.

图像恐惧、缺乏知识、过去对自动化的负面经验、快速变化的代码和遗留代码是自动化的常见障碍。

Fear, lack of knowledge, negative past experiences with automation, rapidly changing code, and legacy code are among the common barriers to automation.

图像自动化回归测试,在自动化构建过程中运行它们,并修复缺陷的根本原因可以减少技术债务并允许可靠代码的增长。

Automating regression tests, running them in an automated build process, and fixing root causes of defects reduces technical debt and permits growth of solid code.

图像自动化回归测试和繁琐的手动任务可以让团队腾出时间从事更重要的工作,例如探索性测试。

Automating regression tests and tedious manual tasks frees the team for more important work, such as exploratory testing.

图像拥有自动化测试和自动化构建流程的团队享有更稳定的速度。

Teams with automated tests and automated build processes enjoy a more stable velocity.

图像如果没有自动回归测试,手动回归测试的范围将继续扩大,最终可能会被忽略。

Without automated regression tests, manual regression testing will continue to grow in scope and eventually may simply be ignored.

图像团队文化和历史可能使程序员更难优先考虑面向业务的测试自动化,而不是编写新功能。使用敏捷原则和价值观有助于整个团队克服测试自动化的障碍。

Team culture and history may make it harder for programmers to prioritize automation of business-facing tests than coding new features. Using agile principles and values helps the whole team overcome barriers to test automation.

第 14 章敏捷测试自动化策略

Chapter 14 An Agile Test Automation Strategy

图像

在我们探索第 III 部分中的每个敏捷测试象限时,我们给出了可以帮助这些不同的测试工作取得成功的工具示例。其中许多工具用于自动化测试。正如我们在上一章中所描述的,团队在寻求成功的测试自动化过程中面临着许多障碍。更好的工具随时可用,但诀窍在于选择正确的工具并学会有效地使用它们。测试自动化需要深思熟虑的投资和渐进式改进。在本章中,我们将解释如何应用敏捷价值观和原则来启动或改进自动化工作。

As we explored each of the Agile Testing Quadrants in Part III, we gave examples of tools that can help those different testing efforts succeed. Many of those tools are for automating tests. As we described in the previous chapter, teams face plenty of obstacles in their quest for successful test automation. Better tools become available all the time, but the trick is to choose the right tools and learn to use them effectively. Test automation requires thoughtful investment and incremental improvement. In this chapter, we explain how you can apply agile values and principles to get traction in starting or improving your automation efforts.

测试自动化的敏捷方法

An Agile Approach to Test Automation

在这里,您正在阅读本章,了解如何让您的测试自动化策略发挥作用,也许您希望找到灵丹妙药,或者您所有问题的答案。我们不想让您失望,但我们需要提前告诉您,没有灵丹妙药。没有一个答案适用于每个团队。不过不要灰心,因为我们有一些想法可以帮助您入门。

Here you are, reading this chapter on how to get your test automation strategy working, maybe hoping for that silver bullet, or an answer to all your questions. We hate to disappoint you, but we need to tell you right up front, there is no silver bullet. There is no one answer that works for every team. Don’t lose heart though, because we have some ideas to help you get started.

首先,我们建议像处理任何问题一样处理您的自动化问题。定义您要解决的问题。为了帮助您理解这一点,我们首先讨论测试自动化的一些基础知识并重新引入一些术语。

First, we suggest approaching your automation problems as you would any problem. Define the problem you are trying to solve. To help you figure that out, we first talk about some basics of test automation and reintroduce some terms.

自动化测试类别

Automation Test Categories

第三部分中,我们介绍了敏捷测试象限,并讨论了每个象限以及每个象限中测试的目的。在本节中,我们从不同的角度来看待象限。让我们仔细看看象限(见图14-1)。

In Part III, we introduced the Agile Testing Quadrants and talked about each quadrant and the purpose of the tests in each quadrant. In this section, we look at the quadrants in a different light. Let’s look carefully at the quadrants (see Figure 14-1).

图 14-1敏捷测试象限

Figure 14-1 Agile Testing Quadrants

图像

您可以看到我们将支持团队的两个象限(Q1 和 Q2)标记为使用自动化。在第四象限,用于从技术角度评价产品的工具通常也需要自动化工具。在第 9 章支持团队的面向业务的测试工具包”中,我们讨论了一些可用于自动化面向业务的测试以支持团队的工具。事实上,唯一没有标记为使用自动化的象限是象限 3——批评产品的面向业务的测试。然而,正如我们在第 10 章中所讨论的,“评价产品的面向业务的测试”,工具可能对某些测试有用。例如,自动化可以帮助设置测试数据和用户场景,以及分析记录的活动。

You can see that we’ve labeled both quadrants that support the team (Q1 and Q2) as using automation. In Quadrant 4, the tools used for critiquing the product from a technology point of view also usually require automated tools. In Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” we discussed some of the tools that can be used for automating business-facing tests in the quest for supporting the team. In fact, the only quadrant that is not labeled as using automation is Quadrant 3—the business-facing tests that critique the product. However, as we discussed in Chapter 10, “Business-Facing Tests that Critique the Product,” tools may be useful for some of that testing. For example, automation can help set up test data and user scenarios, and analyze logged activity.

使用象限来帮助您识别每个项目甚至每个迭代可能需要的不同类型的自动化工具。我们发现遍历每个象限并列出可能需要的工具清单很有帮助。假设我们要重新设计一个 UI。我们看看象限 1。如何编写测试优先的代码?我们知道如何对表示层进行单元测试吗?我们需要一个新工具来帮助解决这个问题吗?现在进入象限 2。我们需要做一些原型制作;我们应该只使用纸张,还是应该计划一个绿野仙踪类型的活动?我们将使用什么工具来创建可执行的面向业务的测试来指导开发?我们是否有需要更新或替换的回归测试脚本?我们知道我们的象限 3 活动之一将是可用性测试。这需要一些提前计划。我们可能需要工具来帮助跟踪用户的活动,以便我们可以进一步分析它们。想想象限 4,我们意识到我们有使用旧 UI 的负载测试脚本,所以我们必须预算时间来更新它们以适应新的 UI。

Use the quadrants to help you identify the different types of automation tools you might need for each project, even for each iteration. We find it helpful to go through each quadrant and make a checklist of what tools might be needed. Let’s say we’re about to redesign a UI. We look at Quadrant 1. How can it be coded test-first? Do we know how to unit test our presentation layer? Do we need a new tool to help with that? Now on to Quadrant 2. We’ll need to do some prototyping; should we just use paper, or should we plan a Wizard of Oz type activity? What tool will we use to create executable business-facing tests to guide development? Do we have regression test scripts that will need updating or replacing? We know that one of our Quadrant 3 activities will be usability testing. That takes some advance planning. We might want tools to help track the users’ activities so we can analyze them further. Thinking about Quadrant 4, we realize that we have load test scripts that use the old UI, so we have to budget time to update them for the new one.

有关 Wizard of Oz 测试的更多信息,请参阅第 8 章支持团队的面向业务的测试”。

See Chapter 8, “Business-Facing Tests that Support the Team,” for more about Wizard of Oz testing.

正如我们在第 III 部分使用敏捷测试象限”中强调的那样,象限的顺序与我们进行测试的顺序无关。当我们为每种类型的测试制作所需的工具清单时,我们会考虑何时进行测试,以便我们知道何时准备好自动化工具。例如,一个设计新架构的团队会计划尽快对其进行峰值测试和可扩展性测试。他们将需要在项目的第一次迭代中花时间寻找和实施性能测试工具。

As we emphasized in Part III, “Using the Agile Testing Quadrants,” the order of quadrants doesn’t relate to the order in which we do the testing. As we make our checklist of tools needed for each type of test, we think about when we want to test so we know when to have our automation tools ready. For example, a team designing a new architecture would plan to do a spike and run scalability test against it as soon as possible. They will need to spend time during the first iteration of the project finding and implementing a performance test tool.

象限帮助我们弄清楚我们可能需要什么工具,但是在不同级别有这么多不同的自动化选项,关于在哪里进行哪种类型的测试以及如何组织测试的策略是必不可少的。为了快速且经常地交付价值,我们的自动化工作需要高投资回报率。测试金字塔帮助我们优化测试投资。

The quadrants help us figure out what tools we might need, but with so many different automation options at different levels, a strategy for where to do which types of testing and how to organize the tests is essential. To deliver value quickly and often, our automation efforts need a high ROI. The test pyramid helps us optimize our test investment.

测试自动化金字塔

Test Automation Pyramid

图 14-2说明了“测试自动化金字塔”。我们喜欢 Mike Cohn 介绍的版本,它显示了由面向技术的单元和组件测试组成的基础层。我们认识到许多团队会为这个想法而苦苦挣扎,因为它似乎与许多团队目前的想法相反。许多测试团队都被教导了“V”测试模型,其中组件、系统和发布测试等活动在编码活动之后按顺序进行。其他团队有一个倒金字塔,大部分测试都在功能层或表示层。

Figure 14-2 illustrates the “Test automation pyramid.” We like the version that Mike Cohn introduced, which shows the foundation layer made up of technology-facing unit and component tests. We recognize that many teams will struggle with this idea, because it seems the opposite of what many teams currently have. Many test teams have been taught the “V” model of testing, where activities such as component, system, and release testing are done in sequence after coding activities. Other teams have an inverted pyramid, with the majority of the tests in the functional or presentation layer.

图 14-2测试自动化金字塔

Figure 14-2 Test automation pyramid

图像

敏捷测试自动化金字塔显示了三个不同的自动化测试层。最低层是支持所有其余部分的基础。它主要由健壮的单元测试和组件测试组成,这些测试是支持团队的面向技术的测试。该层代表大部分自动化测试。它们通常使用与被测系统相同的语言编写,使用 xUnit 系列工具。在一个团队掌握了 TDD 的艺术之后,这些测试是迄今为止编写起来最快、成本最低的。他们也提供最快的反馈,使他们非常有价值。在所有类型的测试中,它们的投资回报率是迄今为止最高的。

The agile test automation pyramid shows three different layers of automated tests. The lowest tier is the foundation that supports all of the rest. It’s mainly made up of robust unit tests and component tests, the technology-facing tests that support the team. This layer represents the bulk of the automated tests. They’re generally written in the same language as the system under test, using the xUnit family of tools. After a team has mastered the art of TDD, these tests are by far the quickest and least expensive to write. They provide the quickest feedback, too, making them highly valuable. They have the biggest ROI by far of any type of test.

在敏捷开发中,我们试图将尽可能多的测试推送到这一层。虽然面向业务的测试倾向于进入更高级别之一,但我们会在有意义时在单元级别实施它们。如果它们是测试,客户不必能够阅读,并且可以像单元测试一样更快地对它们进行编码,这是一个不错的选择。在单元级别也可能进行其他类型的面向技术的测试,例如性能测试。

In agile development, we try to push as many tests as possible to this layer. While business-facing tests tend to go in one of the higher levels, we implement them at the unit level when it makes sense. If they’re tests the customers don’t have to be able to read, and they can be coded much more quickly as unit tests, it’s a good option. Other types of technology-facing tests such as performance tests may also be possible at the unit level.

有关单元和组件测试的更多信息,请参阅第 7 章支持团队的面向技术的测试”。

See Chapter 7, “Technology-Facing Tests that Support the Team” for more about unit and component tests.

金字塔的中间层是包含大多数为支持团队而编写的面向业务的自动化测试的层。这些是功能测试,可以验证我们“正在构建正确的东西”。该层中的测试可能包括“故事”测试、“验收”测试以及涵盖比单元测试层更大的功能集的测试。这些测试在 API 级别或“GUI 后面”运行,直接测试功能而无需通过 GUI。我们编写测试用例,设置输入和固定装置,将输入提供给生产代码,接受输出,并将它们与预期结果进行比较。因为这些测试绕过了表示层,所以与使用接口的测试相比,它们的编写和维护成本更低。

The middle tier in the pyramid is the layer that includes most of the automated business-facing tests written to support the team. These are the functional tests that verify that we are “building the right thing.” The tests in this layer may include “story” tests, “acceptance” tests, and tests that cover larger sets of functionality than the unit test layer. These tests operate at the API level or “behind the GUI,” testing the functionality directly without going through the GUI. We write test cases that set up inputs and fixtures that feed the inputs into the production code, accept the outputs, and compare them to expected results. Because these tests bypass the presentation layer, they are less expensive to write and maintain than tests that use the interface.

有关支持团队的面向业务的测试的更多信息,请参阅第 8 章“支持团队的面向业务的测试”。

See Chapter, 8, “Business-Facing Tests that Support the Team,” for more about business-facing tests that support the team.

我们尝试用客户可以理解的特定领域语言编写它们,因此它们比单元级测试需要更多的工作。它们通常也运行得更慢,因为每个测试比单元测试涵盖更多领域,并且可能访问数据库或其他组件。他们提供的反馈不如单元级测试快,但仍然比我们通过用户界面进行操作要快得多。因此,他们的投资回报率不如构成金字塔底层的测试高,但高于顶层。

We try to write them in a domain-specific language that the customers can understand, so they take more work than unit-level tests. They also generally run more slowly, because each test covers more ground than a unit test and may access the database or other components. The feedback they provide is not as quick as the unit-level tests, but it is still much faster than we could get operating through the user interface. Therefore, their ROI is not as high as the tests that form the base of the pyramid, but it’s higher than the top layer.

Fit 和 FitNesse 是用于金字塔中间层的工具示例。使用电子表格或其他业务友好方式定义测试用例的本土测试工具也很常见。

Fit and FitNesse are examples of tools used for the middle layer of the pyramid. Home-grown test harnesses that use spreadsheets or other business-friendly means for defining test cases are also common.

我们在第 8 章支持团队的面向业务的测试”和第 9 章支持团队的面向业务的测试工具包”中详细介绍了这些测试,我们在其中讨论了支持团队的面向业务的测试以及有效捕获这些测试的工具。

We have more about these tests in Chapter 8, “Business-Facing Tests that Support the Team,” and Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” where we discuss the business-facing tests that support the team and the tools that effectively capture these tests.

顶层代表应该是最小的自动化工作,因为测试通常提供最低的投资回报率。这些测试是通过GUI完成的,是真正操作和操纵表现层的。它们是在代码完成后编写的,因此通常是为了批评产品并直接进入回归套件而编写的。

The top tier represents what should be the smallest automation effort, because the tests generally provide the lowest ROI. These tests are the ones done through the GUI, the ones that actually operate and manipulate the presentation layer. They are written after the code is completed, and so are usually written to critique the product and go directly to the regression suite.

传统上,编写这些测试的成本更高,尽管有新工具可以帮助减少所需的投资。因为用户界面的组件往往会经常更改,所以这些测试比在功能或单元级别工作的测试要脆弱得多。例如,仅仅重命名 HTML 元素可能会导致测试脚本失败。与直接在生产代码上运行的金字塔较低级别的测试相比,通过用户界面进行操作也会减慢这些测试的速度。顶层的测试确实提供了重要的反馈,但是一套 GUI 测试可能需要几个小时才能运行,而不是单元级测试套件所需的几分钟。我们希望尽量减少这一层的测试数量,因此它们应该只构成金字塔的顶端。

These tests are traditionally more expensive to write, although there are new tools that help reduce the investment needed. Because components of the user interface tend to be changed often, these tests are much more brittle than tests that work at a functional or unit level. For example, just renaming HTML elements could cause a test script to fail. Operating through the user interface also slows these tests down, compared to tests in the lower levels of the pyramid that operate directly on production code. The tests in the top layer do provide important feedback, but a suite of GUI tests may take hours to run rather than the few minutes required for unit-level test suites. We want to minimize the number of tests at this layer, so they should only form the tip of the pyramid.

无论有多少自动化测试,大多数系统还需要手动测试活动,例如探索性测试和用户验收测试。我们不想忘记这些,所以我们用金字塔顶端的小云来说明它们。我们的大部分回归测试必须自动化,否则我们的手动测试也不会给我们带来良好的投资回报。

No matter how many automated tests they have, most systems also need manual testing activities, such as exploratory testing and user acceptance testing. We don’t want to forget these, so we’ve illustrated them with the little cloud at the tip of the pyramid. The bulk of our regression testing must be automated or our manual testing won’t give us a good return on investment either.

Patrick Wilson-Welsh [2008] 用“三只小猪”的比喻为测试自动化金字塔添加了一个描述性维度。底部基础层由砖砌成。测试是可靠的,并且不容易受到大灰狼的呼啸而过的影响。中间层由木棍制成。他们需要比砖层更频繁地重新排列以保持坚固。顶层的测试是用稻草做的。很难让它们留在原地,狼很容易将它们吹飞。如果我们有太多用稻草做的测试,我们将花费大量时间将它们恢复原状。

Patrick Wilson-Welsh [2008] adds a descriptive dimension to the test automation pyramid with a “three little pigs” metaphor. The bottom foundation layer is made of bricks. The tests are solid, and not vulnerable to the huffing and puffing of the Big Bad Wolf. The middle layer is made of sticks. They need rearranging more often than the brick layer to stay strong. The tests in the top layer are made of straw. It’s hard to get them to stay in place, and the wolf can easily blow them around. If we have too many tests made out of straw, we’re going to spend lots of time putting them back into shape.

大多数新的敏捷团队并不是从这个形状金字塔开始的——它通常是倒置的,是以前项目的遗留物。GUI 测试工具通常更容易学习,因此团队开始时会在其顶层“稻草”层进行大量测试。正如我们在上一章中提到的,大多数程序员为掌握单元测试自动化而必须克服的“痛苦的驼峰”意味着团队开始时可能只有几块砖。如果系统在设计时考虑到这些测试,那么在中间层自动化功能测试的装置很容易编写,所以木棍可能比积木堆得更快。随着团队掌握 TDD 和单元测试自动化,底层开始增长。当他们获得牵引力时,使用 TDD 的团队将快速构建测试金字塔的砖块基础。

Most new agile teams don’t start with this shape pyramid—it’s usually inverted, a left-over from previous projects. GUI test tools are often easier to learn, so teams start out with a lot of tests in their top “straw” layer. As we mentioned in the previous chapter, the “hump of pain” that most programmers have to overcome to master unit test automation means that the team may start out with only a few bricks. The fixtures that automate functional tests in the middle layer are easy to write if the system is designed with those tests in mind, so the sticks might pile up faster than the bricks. As teams master TDD and unit test automation, the bottom layer starts to grow. When they get traction, a team using TDD will quickly build out the brick foundation of the test pyramid.

请参阅参考书目以获取指向 Patrick Wilson-Welsh 关于“翻转测试金字塔”的讨论的链接。

See the bibliography for a link to Patrick Wilson-Welsh’s discussion of “flipping the test pyramid” right-side up.

测试金字塔是开始研究测试自动化如何帮助敏捷团队的好地方。程序员往往专注于金字塔的底部,他们需要大量的时间和培训来克服“痛苦的驼峰”并达到 TDD 自然而快速的地步。在传统团队中,测试人员通常别无选择,只能在 GUI 级别进行自动化测试。敏捷团队使用的全团队方法意味着测试人员与程序员结对并帮助他们更好地编写测试,这反过来又巩固了金字塔的基础层。因为测试驱动开发,整个团队总是在设计最大的可测试性,金字塔可以成长为正确的形状。

The testing pyramid is a good place to start looking at how test automation can help an agile team. Programmers tend to focus on the bottom of the pyramid, and they need plenty of time and training to get over the “hump of pain” and get to the point where TDD is natural and quick. In traditional teams, testers usually have no choice but to automate tests at the GUI level. The whole-team approach used by agile teams means that testers pair with programmers and help them get better at writing tests, which in turn solidifies that brick foundation layer of the pyramid. Because tests drive development, the whole team is always designing for maximum testability, and the pyramid can grow to the right shape.

程序员与测试人员配对以自动化功能级测试,填充中间层。例如,测试人员和客户可能会为 Web 服务应用程序准备 400 行的测试用例电子表格。程序员可以帮助找到一种自动化这些测试的方法。不同的团队成员可能在生成测试数据或使用 Excel 宏等工具等领域拥有专业知识,并且所有这些知识都在团队中传播。团队一起工作,找到工具、测试用例和测试数据的最佳组合。

Programmers pair with testers to automate functional-level tests, filling out the middle layer. For example, a tester and customer may prepare a 400-row spreadsheet of test cases for a web services application. The programmer can help figure out a way to automate those tests. Different team members may have expertise in areas such as generating test data or using tools such as Excel macros, and all that knowledge spreads around the team. Working together, the team finds the best combinations of tools, test cases, and test data.

让程序员参与寻找具有成本效益的方法来自动化顶层 GUI 测试有很多好处。这些努力可以让程序员更好地理解系统的“大局”,测试人员可以学习如何创建更灵活、更简单的 GUI 测试。

Involving the programmers in finding cost-effective ways to automate the top-level GUI tests has multiple benefits. These efforts may give programmers a better understanding of the system’s “big picture,” and testers can learn how to create more pliable, less straw-like GUI tests.

团队合作和共享知识越多,团队、应用程序和测试就会变得越强大。大灰狼不会有机会的。让我们首先看看我们可以自动化什么样的测试,然后看看我们甚至不应该尝试的测试。

The more a team can work together and share knowledge, the stronger the team, the application, and the tests will become. The Big Bad Wolf won’t stand a chance. Let’s start by looking at what kind of tests we can automate and then at what we shouldn’t even try.

我们可以自动化什么?

What Can We Automate?

您能想到的大多数类型的测试都受益于自动化。手动单元测试对于防止回归失败没有多大帮助,因为在每次签入之前执行一套手动测试是不切实际的。您也不能通过手动单元测试来设计测试先行的代码。当程序员不能按一下按钮快速运行测试,他们可能根本没有足够的动力来运行测试。我们可以手动测试不同的代码单元是否可以正确地协同工作,但是自动化组件测试是一个更有效的安全网。

Most types of testing you can think of benefit from automation. Manual unit tests don’t go far toward preventing regression failures, because performing a suite of manual tests before every check-in just isn’t practical. You can’t design code test-first through manual unit tests either. When programmers can’t run tests quickly at the touch of a button, they may not be motivated enough to run tests at all. We could manually test that different units of code work together correctly, but automated component tests are a much more effective safety net.

手动探索性测试是发现功能缺陷的有效方法,但如果我们没有足够的自动化面向业务的回归测试,我们可能会把所有时间都花在疯狂地试图跟上手动回归测试。让我们谈谈可以通过自动化很好地完成的所有不同类型的测试。

Manual exploratory testing is an effective way to find functional defects, but if we don’t have enough automated business-facing regression tests, we probably spend all of our time madly trying to keep up with manual regression testing. Let’s talk about all of the different kinds of testing that can be done well with automation.

要运行自动化测试,您需要某种自动化框架,允许程序员经常签入代码、对该代码运行测试并创建可部署文件。让我们先考虑一下。

To run automated tests, you need some kind of automated framework that allows programmers to check in code often, run tests on that code, and create deployable files. Let’s consider this first.

持续集成、构建和部署

Continuous Integration, Builds, and Deploys

软件开发中涉及的任何繁琐或重复性任务都适合自动化。我们已经讨论了自动化构建过程的重要性。没有这个你就无法构建你的自动化测试金字塔。您的团队需要来自单元级测试的即时反馈以保持在正轨上。获取自动生成的电子邮件,列出每个签入的更改,这对测试人员有很大帮助,因为他们知道什么时候生成可以进行测试,而不必打扰程序员。

Any tedious or repetitive task involved in developing software is a candidate for automation. We’ve talked about the importance of an automated build process. You can’t build your automated test pyramid without this. Your team needs the immediate feedback from the unit-level tests to stay on track. Getting automated build emails listing every change checked in is a big help to testers because they know when a build is ready to test without having to bother the programmers.

有关构建自动化工具的示例,请参阅第 7 章支持团队的面向技术的测试”。

See Chapter 7, “Technology-Facing Tests that Support the Team,” for examples of build automation tools.

危险:等待周二的构建

Peril: Waiting for Tuesday’s Build

在传统环境中,测试人员等待稳定构建是正常的,即使这意味着要等到下周二。在敏捷环境中,如果测试人员跟不上开发人员,故事就会在游戏后期进行测试。如果开发人员没有得到反馈,例如建议和错误,测试人员可能会失去开发人员的信任。错误不会被发现,直到开发人员已经在另一个故事上并且不想被打断以修复它们,直到以后。

In a traditional environment, it is normal for testers to wait for a stable build, even if that means waiting until next Tuesday. In an agile environment, if testers don’t keep up with the developers, the stories get tested late in the game. If the developers don’t get the feedback, such as suggestions and bugs, the testers can lose credibility with the developers. Bugs won’t be discovered until the developers are already on another story and do not want to be interrupted to fix them until later.

错误堆积,自动化因为无法完成而受到影响。速度会受到影响,因为故事在经过测试之前无法标记为“完成”。这使得计划下一次迭代变得更加困难。在发布周期结束时,您的故事测试运行到最终游戏,您可能无法成功发布。至少,你会有一个压力释放。

Bugs pile up, and automation suffers because it can’t be completed. Velocity is affected because a story cannot be marked “done” until it is tested. This makes it harder to plan the next iteration. At the end of the release cycle, your story testing runs into the end game and you may not have a successful release. At the very least, you will have a stressful release.

自动化部署过程还可以加快测试速度并减少错误。事实上,珍妮特在编辑本章的那天,她搞砸了部署,因为这是一个手动过程。这很简单,但她是这个项目的新手,将文件移到了错误的地方。在 Janet 的清单上列出了需要立即完成的事情,将自动部署流程安排到位。Lisa 的团队首先实施了其持续集成和构建框架,发现这相当容易和快速,尽管它需要持续的关注和支持。其他团队,尤其是那些拥有大型复杂系统的团队,面临着更大的障碍。

An automated deployment process also speeds up testing and reduces errors. In fact, the day Janet was editing this chapter, she messed up the deployment because it was a manual process. It was pretty simple, but she was new to the project and moved the file to the wrong place. Getting an automated deployment process in place went on Janet’s list of things to get done right away. Lisa’s team implemented its continuous integration and build framework first thing, and found it fairly easy and quick to do, although it requires continual care and feeding. Other teams, especially those with large, complex systems, face much bigger hurdles.

我们与构建时间为两个小时或更长时间的团队进行了交谈。这意味着程序员在签入代码后必须等待两个小时才能验证他的签入没有破坏任何预先存在的功能。那是一个漫长的等待时间。

We’ve talked with teams who had build times of two hours or more. This meant that a programmer would have to wait for two hours after checking in code to get validation that his check-in didn’t break any preexisting functionality. That is a long time to wait.

大多数敏捷团队发现持续构建时间超过 8 到 10 分钟是不可行的。即使是 15 分钟对于等待反馈来说也太长了,因为签到将开始堆积,测试人员将等待很长时间才能获得最新、最好的构建。您能想象开发人员在接近迭代或发布周期结束时处理一个需要两个小时的构建的感觉吗?如果他们破坏了任何功能,他们将不得不再等两个小时才能了解他们是否修复了它。

Most agile teams find an ongoing build longer than eight to ten minutes to be unworkable. Even 15 minutes is much too long to wait for feedback, because check-ins will start stacking up, and testers will wait a long time to get the latest, greatest build. Can you imagine how the developers working with a build that takes two hours feel as they approach the end of an iteration or release cycle? If they break any functionality, they’ll have to wait two more hours to learn whether or not they had fixed it.

很多时候,长时间构建是访问数据库或尝试通过界面进行测试的结果。针对大型代码库运行的数千个测试会对运行构建的机器资源造成负担。对您的测试进行一些分析,看看瓶颈在哪里。例如,如果是数据库访问导致了大部分问题,请尝试模拟真实数据库并改用内存中的数据库。配置构建过程以在多台机器上分发测试。查看不同的软件是否可以帮助更好地管理资源。如果需要,请从您的团队外部聘请专家提供帮助。

Many times, long builds are the result of accessing the database or trying to test through the interface. Thousands of tests running against a large codebase can tax the resources of the machine running the build. Do some profiling of your tests and see where the bottleneck is. For example, if it is the database access that is causing most of the problems, try mocking out the real database and use an in-memory one instead. Configure the build process to distribute tests across several machines. See if different software could help manage resources better. Bring in experts from outside your team to help if needed.

加速持续集成和构建过程的关键是一次一小步。一次引入一个更改,以便您可以分别衡量每个成功并知道您在正确的轨道上。首先,您可能希望简单地删除成本最高(就时间而言)的测试以每晚运行而不是在每次构建时运行。

The key to speeding up a continuous integration and build process is to take one small step at a time. Introduce changes one at a time so that you can measure each success separately and know you are on the right track. To start with, you may want to simply remove the most costly (in terms of time) tests to run nightly instead of on every build.

快速运行的持续集成和构建过程可为任何自动化工作提供最大的投资回报率。这是每个团队需要自动化的第一件事。当它到位时,团队就有办法从自动化测试中获得快速反馈。接下来,我们看看应该自动化的不同类型的测试。

A fast-running continuous integration and build process gives the greatest ROI of any automation effort. It’s the first thing every team needs to automate. When it’s in place, the team has a way to get quick feedback from the automated tests. Next, we look at different types of tests that should be automated.

请参阅参考书目以获取构建自动化工具的链接和包含有关改进构建过程的更多信息的书籍。

See the bibliography for links to build automation tools and books with more information about improving the build process.

单元和组件测试

Unit and Component Tests

我们怎么强调自动化单元测试的重要性都不为过。如果您的程序员使用 TDD 作为一种机制来编写他们的测试,那么他们不仅在创建一个很棒的回归套件,而且还在使用它们来设计高质量、健壮的代码。如果您的团队没有自动化单元测试,那么其长期成功的机会就很小。将单元级测试自动化和持续集成作为您的首要任务。

We can’t overemphasize the importance of automating the unit tests. If your programmers are using TDD as a mechanism to write their tests, then they are not only creating a great regression suite, but they are using them to design high-quality, robust code. If your team is not automating unit tests, its chances of long-term success are slim. Make unit-level test automation and continuous integration your first priority.

第 7 章,“支持团队的面向技术的测试”,详细介绍了一些可以使用的工具。

Chapter 7, “Technology-Facing Tests that Support the Team,” goes into detail about some of the tools that can be used.

API 或网络服务测试

API or Web Services Testing

使用某种形式的自动化最容易测试 API 或 Web 服务应用程序。Janet 所在的团队成功地使用 Ruby 读取包含输入变量的所有排列和组合的电子表格,并将输出与存储在电子表格中的预期结果进行比较。这些数据驱动的测试易于编写和维护。

Testing an API or web services application is easiest using some form of automation. Janet has been on teams that have successfully used Ruby to read in a spreadsheet with all of the permutations and combinations of input variables and compare the outputs with the expected results stored in the spreadsheets. These data-driven tests are easy to write and maintain.

Janet 的一位客户使用 Ruby 的 IRB(交互式 Ruby Shell)功能来测试 Web 服务以进行验收测试。该团队愿意与客户团队共享其脚本,但业务测试人员更愿意观察如果即时更改输入会发生什么。以半自动方式交互式运行测试允许这样做。

One customer of Janet’s used Ruby’s IRB (Interactive Ruby Shell) feature to test the web services for acceptance tests. The team was willing to share its scripts with the customer team, but the business testers preferred to watch to see what happened if inputs were changed on the fly. Running tests interactively in a semiautomated manner allowed that.

GUI 背后的测试

Testing behind the GUI

GUI 背后的测试比 GUI 本身的测试更容易实现自动化。因为测试不受表示层更改的影响,并且在更稳定的业务逻辑代码上工作,所以它们更稳定。用于此类测试的工具通常使用表格或电子表格以声明性格式编写测试。获取生产代码以对测试输入进行操作并返回结果的固定装置通常可以快速编写。这是编写面向业务的测试的主要领域,客户和推动开发的开发人员都可以理解。

Testing behind the GUI is easier to automate than testing the GUI itself. Because the tests aren’t affected by changes to the presentation layer and work on more stable business logic code, they’re more stable. Tools for this type of testing typically provide for writing tests in a declarative format, using tables or spreadsheets. The fixtures that get the production code to operate on the test inputs and return the results can generally be written quickly. This is a prime area for writing business-facing tests, understandable to both customers and developers that drive development.

测试图形用户界面

Testing the GUI

即使是只有很少或没有业务逻辑的瘦 GUI 也需要进行测试。敏捷开发的快速步伐,每次迭代都交付新功能,要求在 GUI 级别对大多数项目进行一些自动化回归测试。工具选择是成功实现 GUI 自动化的关键。自动化脚本需要灵活且易于维护。当使用良好的编码实践开发框架时,Janet 非常成功地使用了 Ruby 和 Watir,就好像它是一个生产应用程序一样。开发库的时间被投入了,这样代码中就没有太多的返工或重复,并且可以在一个地方进行所需的更改。使代码易于维护可以增加这些测试的投资回报率。

Even a thin GUI with little or no business logic needs to be tested. The fast pace of agile development, delivering new functionality each iteration, mandates some automated regression tests at the GUI level for most projects. Tool selection is key for successful GUI automation. The automated scripts need to be flexible and easy to maintain. Janet has used Ruby and Watir very successfully when the framework was developed using good coding practices, just as if it were a production application. Time was put into developing the libraries so that there was not a lot of rework or duplication in the code, and changes needed could be made in one place. Making the code easy to maintain increased the ROI on these tests.

这里有一点关于可测试性——确保程序员命名他们的对象或为他们分配 ID。如果它们依赖于系统生成的标识符,那么每次将新对象添加到页面时,ID 都会发生变化,从而需要对测试进行更改。

A point about testability here—make sure the programmers name their objects or assign IDs to them. If they rely on system-generated identifiers, then every time a new object is added to the page, the IDs will change, requiring changes to the tests.

有关 GUI 测试框架的示例,请参阅第 9 章支持团队的面向业务的测试工具包”。

See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for examples of GUI test frameworks.

只对实际界面进行测试。检查一些事情,比如确保按钮真正起作用并做它们应该做的事情。不要试图测试业务功能。其他可以轻松自动化的测试类型是链接检查器。无需有人手动检查每个页面上的每个链接以确保他们访问了正确的页面。寻找唾手可得的果实,先将容易实现自动化的事情自动化,这样您就会有更多时间应对更大的挑战。

Keep the tests to just the actual interface. Check things like making sure the buttons really work and do what they are supposed to. Don’t try to try to test business functionality. Other types of tests that can be automated easily are link checkers. There is no need for someone to manually go through every link on every page to make sure they hit the right page. Look for the low-hanging fruit, automate the things that are simple to automate first, and you’ll have more time for the bigger challenges.

负载测试

Load Tests

没有自动化就无法完成某些类型的测试。手动负载测试通常不可行或不准确,尽管我们都曾经或多次尝试过。性能测试既需要监控工具,也需要一种在被测系统中驱动操作的方法。如果没有一些工具框架,您无法生成高容量攻击来验证网站是否可以被黑客入侵或是否可以处理大量负载。

Some types of testing can’t be done without automation. Manual load tests aren’t usually feasible or accurate, although we’ve all tried it at one time or another. Performance testing requires both monitoring tools and a way to drive actions in the system under test. You can’t generate a high-volume attack to verify whether a website can be hacked or can handle a large load without some tool framework.

有关负载测试自动化工具的示例,请参阅第 11 章使用面向技术的测试来评价产品”。

See Chapter 11, “Critiquing the Product Using Technology-Facing Tests,” for examples of load test automation tools.

比较

Comparisons

如果您首先解析文件并以人类可读的格式显示它,那么直观地检查系统进程输出的 ASCII 文件会容易得多。用于比较输出文件以确保没有进行无意更改的脚本比尝试手动比较它们更快、更准确。文件比较工具比比皆是,从免费的 diff 到 WinDiff 等专有工具。源代码管理工具和 IDE 都有自己的内置比较工具。这些是每个测试人员工具箱中必不可少的项目。在为数据仓库或数据迁移项目进行测试时,不要忘记创建用于比较数据库表的脚本。

Visually checking an ASCII file output by a system process is much easier if you first parse the file and display it in a human-readable format. A script to compare output files to make sure no unintentional changes were made is a lot faster and more accurate than trying to compare them manually. File comparison tools abound, ranging from the free diff to proprietary tools such as WinDiff. Source code management tools, and IDEs have their own built-in comparison tools. These are essential items in every tester’s toolbox. Don’t forget about creating scripts for comparing database tables when doing testing for your data warehouse or data migration projects.

在第 7 章支持团队的面向技术的测试”中阅读有关源代码管理工具和 IDE 的更多信息。

Read more about source code management tools and IDEs in Chapter 7, “Technology-Facing Tests that Support the Team.”

重复性任务

Repetitive Tasks

当我们与客户合作以更好地了解业务并了解对他们有价值的东西时,我们可能会看到将他们的某些任务自动化的机会。丽莎的公司需要向所有客户邮寄几份带有求职信的表格。程序员不仅可以生成表格,还可以将它们与求职信连接起来,大大加快了邮寄速度。Lisa 的测试伙伴 Mike Busse 编写了一个电子表格宏来进行复杂的资金分配计算,而退休计划管理员一直在手动进行这些计算。许多手动清单可以用自动化脚本代替。自动化不仅仅用于测试。

As we work with our customers to better understand the business and learn what’s valuable to them, we might see opportunities to automate some of their tasks. Lisa’s company needed to mail several forms with a cover letter to all of their clients. The programmers could not only generate the forms but could also concatenate them with the cover letter and greatly speed up the mailing effort. Lisa’s fellow tester, Mike Busse, wrote a spreadsheet macro to do complex calculations for allocating funds that the retirement plan administrators had been doing manually. A lot of manual checklists can be replaced with an automated script. Automation isn’t just for testing.

数据创建或设置

Data Creation or Setup

另一个有用的自动化领域是数据创建或设置。如果您经常设置数据,请自动化该过程。通常,我们需要多次重复某件事才能重现错误。如果这可以自动化,你将保证每次都有相同的结果。

Another useful area for automation is data creation or setup. If you are constantly setting up your data, automate the process. Often, we need to repeat something multiple times to be able to recreate a bug. If that can be automated, you will be guaranteed to have the same results each time.

丽莎的故事

Lisa’s Story

我们的许多测试模式,包括用于自动回归套件的测试模式,都使用规范数据。这种规范或“种子”数据最初是从生产中获取的。数据库中的某些表(例如查找表)不会更改,因此永远不需要用新副本刷新它们。每当回归套件运行时,其他表(例如包含退休计划、员工和交易信息的表)都需要从零开始。

Many of our test schemas, including the ones used for automated regression suites, use canonical data. This canonical or “seed” data was originally taken from production. Some tables in the database, such as lookup tables, don’t change, so they never need to be refreshed with a new copy. Other tables, such as those containing retirement plan, employee, and transaction information, need to start from Ground Zero whenever a regression suite runs.

我们的数据库开发人员编写了一个存储过程来刷新“种子”模式中的每个测试模式。我们的测试人员可以在一个名为 REFRESH_TABLE_LIST 的特殊表中指定我们想要刷新的表。我们为每个测试模式都有一个 ant 目标来运行刷新数据的存储过程。自动构建使用这个目标,但每当我们想要清理测试模式并重新开始时,我们都会自己使用它。

Our database developer wrote a stored procedure to refresh each test schema from the “seed” schema. We testers may specify the tables we want refreshed in a special table called REFRESH_TABLE_LIST. We have an ant target for each test schema to run the stored procedure that refreshes the data. The automated builds use this target, but we use it ourselves whenever we want to clean up our test schema and start over.

我们的许多回归测试都在“种子”数据之上创建了自己的数据。我们的 Watir 测试创建了他们需要的所有数据,并包含使它们无论存在什么数据都可以重新运行的逻辑。例如,测试员工从他或她的退休计划中申请贷款的脚本首先会取消所有现有贷款,以便可以提取新贷款。

Many of our regression tests create their own data on top of the “seed” data. Our Watir tests create all of the data they need and include logic that makes them rerunnable no matter what data is present. For example, the script that tests an employee requesting a loan from his or her retirement plan first cancels any existing loans so a new one can be taken out.

测试数据库层的 FitNesse 测试也会创建自己的数据。我们使用一个特殊的模式,其中我们已经删除了大部分约束,因此我们不必添加每个表的每一列。测试仅添加与被测试功能相关的数据。每个测试都会删除它创建的数据,因此后续测试不会受到影响,并且每个测试都是独立且可重新运行的。

FitNesse tests that test the database layer also create their own data. We use a special schema where we have removed most constraints, so we don’t have to add every column of every table. The tests only add the data that’s pertinent to the functionality being tested. Each test tears down the data it created, so subsequent tests aren’t affected, and each test is independent and rerunnable.

——丽莎

—Lisa

清理测试数据与生成测试数据一样重要。您的数据创建工具包应该包括拆除测试数据的方法,这样它就不会影响不同的测试或阻止重新运行相同的测试。

Cleaning up test data is as important as generating it. Your data creation toolkit should include ways to tear down the test data so it doesn’t affect a different test or prevent rerunning the same test.

我们已经研究了需要或至少有用自动化的主要领域。我们的意见是,每当您需要进行测试或一些与测试相关的活动时,首先要确定是否可以通过自动化来辅助。在某些情况下,自动化并不合适。让我们看看其中的一些。

We’ve looked at major areas where automation is required or at least useful. Our opinion is that whenever you need to do a test or some testing-related activity, first decide whether it can be aided by automation. In some cases, automation won’t be appropriate. Let’s look at some of those.

我们不应该自动化什么?

What Shouldn’t We Automate?

有些测试需要人的眼睛、耳朵和智慧。可用性和探索性测试属于这一类。其他可能无法证明自动化投资合理的测试是一次性测试和永远不会失败的测试。

Some testing needs human eyes, ears, and intelligence. Usability and exploratory testing are two that fall into that category. Other tests that may not justify the automation investment are one-off tests and those that will never fail.

可用性测试

Usability Testing

真正的可用性测试需要有人实际使用该软件。自动化可能有助于设置场景以随后检查可用性。观察用户的行为,向他们汇报他们的体验,并判断结果是了解软件可用性方面无法自动化的人的工作。记录用户操作有助于可用性测试。

Real usability testing requires someone to actually use the software. Automation might be helpful in setting up scenarios to subsequently examine for usability. Observing users in action, debriefing them on their experiences, and judging the results is a job for a person who understands that usability aspects of software cannot be automated. Logging user actions is helpful for usability testing.

我们在第 10 章评价产品的面向业务的测试”中讨论了一些日志记录和监控工具。

We discuss some logging and monitoring tools in Chapter 10, “Business-Facing Tests that Critique the Product.”

珍妮特的故事

Janet’s Story

我们评估了几种 GUI 工具,但决定将 Ruby 与 Watir 结合使用。我们将测试仅限于 GUI 功能。我们的一项测试是检查以确保正确的验证消息显示在屏幕上。我正在运行测试,碰巧正在看屏幕,因为我没有看到其他测试人员创建的这个特定测试。我的眼睛发现了一些奇怪的东西,但测试通过了,所以我又重播了一遍。其中一位程序员在屏幕上添加了一个“$”,错误消息因此被偏移显示。显示了正确的消息,只是不在正确的位置。在这种情况下,观察测试运行的价值是巨大的,因为我们准备很快发布,我们可能不会发现那个特定的问题。

We had evaluated several GUI tools but decided to use Ruby with Watir. We kept our tests limited to GUI functions only. One of our tests was checking to make sure that correct validation messages were displaying on the screen. I was running the tests and happened to be watching the screen because I hadn’t seen this particular test that one of the other testers created. My eyes caught something weird, but the test passed, so I replayed it again. One of the programmers had added a “$” to the screen, and the error message was displayed offset because of it. The correct message was displayed, just not in the right place. In this instance, the value in watching the tests run was huge because we were preparing to release fairly soon, and we probably wouldn’t have caught that particular problem.

——珍妮特

—Janet

可以通过自动化测试来确保 GUI 永不更改,但您需要问问自己是否值得为此付出代价。你真的关心一个按钮改变了一个像素的位置吗?结果证明努力是正确的吗?我们认为您不应该自动化“外观和感觉”测试,因为自动化脚本只能查找您告诉它要查看的内容。自动化会错过会跳到人类身上的视觉问题。

It is possible to automate tests that make sure the GUI never changes, but you need to ask yourself whether it’s worth the cost. Do you really care that a button has changed positions by one pixel? Do the results justify the effort? We don’t think you should automate “look and feel” testing, because an automated script can only look for what you tell it to see. Automation would miss visual problems that would jump out at a human.

探索性测试

Exploratory Testing

同样,探索性测试可以通过脚本来加速以创建测试数据并跳过一些设置步骤,但它需要熟练的测试人员来设计和执行测试。探索性测试的一个主要目标是通过实践更多地了解产品,然后使用该信息改进未来的开发。自动化脚本不会为您做这些。但是,正如我们之前所说,如果没有大量其他自动化测试,您将没有时间进行探索性测试。

Similarly, exploratory testing may be speeded up with scripts to create test data and jump through some setup steps, but it requires a skilled tester to design and execute the tests. One major goal of exploratory testing is to learn more about the product by doing, and then use that information to improve future development. Automated scripts won’t do that for you. However, as we’ve said before, you won’t have time for exploratory testing without a lot of other automated tests.

请参阅第 10 章,“评价产品的面向业务的测试”,了解更多关于探索性测试和可以促进它的工具。

See Chapter 10, “Business-Facing Tests that Critique the Product,” for more on exploratory testing and tools that can facilitate it.

永远不会失败的测试

Tests that Will Never Fail

我们听说过这样一种说法,即永远不会失败的测试不需要自动化。如果一个需求如此明显以至于只有一种方法可以实现它,并且没有程序员会在不知道它应该做什么的情况下查看该代码,那么有人在该代码中引入缺陷的可能性几乎为零。假设我们有一个带有地址字段的表单。我们是否需要自动回归测试来验证不需要第二条街道地址行?在我们手动验证之后,以后有人不小心将其更改为必填字段的可能性有多大?即使有人这样做,也不会是灾难性的事件。其他人会注意到它,人们可以轻松解决它,直到它被修复。

We’ve heard an argument that tests that will never fail don’t need to be automated. If a requirement is so obvious that there’s only one way to implement it, and no programmer will ever look at that code later without knowing exactly what it should do, the chances of someone introducing a defect in that code are next to nothing. Let’s say we have a form with address fields. Do we need an automated regression test to verify that the second street address line is not required? After we’ve verified it manually, how likely is it that someone will accidentally change it to a required field later? Even if someone did, it wouldn’t be a catastrophic event. Someone else would notice it and people could work around it easily until it was fixed.

话又说回来,对它的测试很容易包括在内。诸如复制/粘贴错误之类的程序员技巧一直在发生。如果您觉得一次性手动测试就可以完成工作并且未来失败的风险不足以证明自动化回归测试是合理的,请不要将它们自动化。如果您的决定被证明是错误的,您以后将有另一个机会使它们自动化。如果您不确定,并且自动化并不难,那就去做吧。

Then again, a test for it would be easy to include. And programmer tricks such as copy/paste errors happen all the time. If you feel comfortable that one-time manual testing does the job and that the risk of future failures doesn’t justify automating regression tests, don’t automate them. If your decision turns out to be wrong, you’ll get another chance to automate them later. If you aren’t sure, and it’s not terribly difficult to automate, go for it.

如果您正在测试一个生命攸关的系统,即使是非常小的回归失败风险也太大了。使用风险分析来帮助决定哪些测试应该自动化。

If you’re testing a life-critical system, even a very small risk of a regression failure is too much. Use risk analysis to help decide what tests should be automated.

有关风险分析及其与测试的关系的更多信息,请参阅第 18 章编码和测试”。

See Chapter 18, “Coding and Testing,” for more about risk analysis and how it relates to testing.

一次性测试

One-Off Tests

大多数时候,手动执行一次性测试就足够了。如果自动化测试没有回报,为什么要这么做?有时自动化对于一次性测试来说是值得的。

Most times, manually executing a one-off test is sufficient. If automating a test doesn’t have payoff, why do it? Sometimes automation is worth doing for a one-off test.

丽莎的故事

Lisa’s Story

我们最近做了一个故事,在发布工资单时弹出警告消息对话框,但该消息应该只在 1 月的前两周出现。对此功能的自动化测试需要某种方法来模拟当前日期在 1 月 1 日到 1 月 15 日之间。这并不难做到,但失败的后果相当微不足道,我们有更多关键的故事可以提供迭代。当时自动化该测试没有足够的价值来证明成本是合理的,而且风险因素很低。我们决定手动测试它。

We recently did a story to pop up a warning message dialog when posting a payroll, but the message should only come up during the first two weeks of January. Automating a test for this functionality would require some way to simulate that the current date was between January 1 and January 15. That’s not terribly hard to do, but the consequences of a failure were fairly trivial, and we had more critical stories to deliver that iteration. Automating that test at that time just didn’t have enough value to justify the cost, and the risk factor was low. We decided to test it manually.

在其他情况下,进行一次性测试似乎是最直观的,但自动化是更好的选择。我们为不同的业务合作伙伴托管站点,每个站点都有独特的内容、外观和感觉。数据库中的值驱动每个品牌的正确行为和内容。一些数据非常复杂,例如基于资产价值和参与者数量的费用表。使用 FitNesse 测试来验证此数据要容易得多,也准确得多。我们有一组固定装置,可以让我们为要测试的合作伙伴“品牌”指定键。我们可以轻松地从业务开发人员为每个新合作伙伴创建的电子表格中插入适当的预期结果。这些测试不是我们回归套件的一部分。它们仅用于验证新品牌一次。

There are other cases where doing a one-off test seems the most intuitive but automation is a better choice. We host sites for different business partners, and each one has unique content, look, and feel. Values in the database drive the correct behavior and content for each brand. Some of the data, such as fee schedules based on asset values and numbers of participants, are highly complex. It’s much easier and much more accurate to verify this data using FitNesse tests. We have a set of fixtures that let us specify keys for the partner “brand” that we want to test. We can easily plug in the appropriate expected results from the spreadsheets that the business development staff creates for each new partner. These tests aren’t part of our regression suite. They’re used one time only to validate the new brand.

——丽莎

—Lisa

繁琐的任务可能值得自动化,即使您不经常这样做。权衡自动化成本与手动执行测试所消耗的宝贵时间。如果它很容易手动完成,而自动化不会很快,那就保持手动。

Tedious tasks may be worth automating, even if you don’t do them often. Weigh the automation cost against the amount of valuable time eaten up by manually doing the test. If it’s easy to do manually, and automating wouldn’t be quick, just keep it manual.

什么可能难以自动化?

What Might Be Hard to Automate?

如果代码不是以测试为先编写的,或者至少没有考虑到测试自动化,那么自动化就更难了。较旧的系统往往属于这一类,但毫无疑问,仍在生产大量具有相同不可测试特性的新代码。

When code isn’t written test-first, or at least with test automation in mind, it’s much harder to automate. Older systems tend to fall into this category, but no doubt plenty of new code with the same untestable characteristics is still being produced.

如果您面临处理尚未进行自动化测试的现有代码,那么您将面临一场艰苦的战斗,但这是一场可以取胜的战斗。遗留代码可能有 I/O、数据库访问以及业务逻辑和表示代码交织在一起。可能不清楚在何处挂钩代码以自动执行测试。您如何开始在这样的系统上进行自动化测试?您当然不能计划自动化 GUI 下的所有内容,因为大部分逻辑都在表示层中。

If you’re faced with working on existing code that doesn’t already have automated tests, you’re in for an uphill battle, but a winnable one. Legacy code may have I/O, database access, as well as business logic and presentation code intertwined. It may not be clear where to hook into the code to automate a test. How do you get started automating tests on such a system? You certainly can’t plan on automating everything below the GUI, because much of the logic is in the presentation layer.

至少有几种不同的方法行之有效。我们在第 13 章我们为什么要自动化测试以及是什么阻碍了我们”中谈到的“痛苦的驼峰”令人生畏,但它是可以克服的,然后测试自动化将变得容易得多。Michael Feathers 的Working Effectively With Legacy Code [2004] 解释了如何围绕现有代码库构建测试工具并重构它们以适应自动化。即使使用遗留代码,您也可以编写测试来防止引入新问题。这种方法甚至适用于缺乏结构或非面向对象的系统。

There are at least a couple of different approaches that work well. The “hump of pain” we talked about in Chapter 13, “Why We Want to Automate Tests and What Holds Us Back,” is intimidating, but it can be overcome, and then test automation will become much easier. Michael Feathers’ Working Effectively With Legacy Code [2004] explains how to build a test harness around existing code bases and refactor them to accommodate automation. Even with legacy code, you can write tests to protect against introducing new problems. This approach can work even on systems that lack structure or aren’t object-oriented.

Lisa 的团队决定采用一种不同但同样有效的方法。团队成员开始通过在新的测试友好架构中编写所有新功能来“扼杀”遗留代码。他们逐渐用测试优先编写的代码替换所有旧代码。当他们确实处理旧代码以修复错误时,或者在旧代码需要更新的情况下,他们只需为他们更改的所有代码添加单元测试。GUI 冒烟测试套件涵盖了没有单元测试的遗留系统其余部分的关键功能。

Lisa’s team decided on a different but equally effective approach. The team members started “strangling” the legacy code by writing all new features in a new test-friendly architecture. They’re gradually replacing all of the old code with code written test-first. When they do work on old code to fix bugs, or in the cases where the old code needs updating, they simply add unit tests for all of the code they change. A GUI smoke test suite covers the critical functions of the rest of the legacy system that has no unit tests.

第 7 章,“支持团队的面向技术的测试”,详细介绍了处理遗留代码的不同敏捷方法。

Chapter 7, “Technology-Facing Tests that Support the Team,” goes into more detail about different agile approaches to legacy code.

与任何自动化项目一样,一次处理一个难以自动化的代码,并首先解决风险最高的区域。解决可测试性问题并找到编写单元级测试的方法。努力会有回报。

As with any automation project, approach the hard-to-automate code one piece at a time, and address the highest risk areas first. Solve the testability problem and find a way to write unit-level tests. The effort will pay off.

制定自动化战略——我们从哪里开始?

Developing an Automation Strategy—Where Do We Start?

一个简单的、循序渐进的方法听起来与自动化策略不相容,但在敏捷测试中,我们试图首先理解问题。决定从哪里以及如何开始自动化需要一些思考和讨论。当您的团队着眼于测试挑战时,您将需要考虑自动化适合的地方。在开始搜索特定的自动化工具之前,您需要确定您的需求。

A simple, step-by-step approach sounds incompatible with an automation strategy, but in agile testing we try to understand the problem first. Deciding where and how to start with automation requires a bit of thought and discussion. As your team looks at testing challenges, you’ll need to consider where automation is appropriate. Before you start searching for a particular automation tool, you’ll want to identify your requirements.

您需要了解您要解决的问题是什么。你想自动化什么?例如,如果您没有任何类型的测试自动化,并且您开始购买昂贵的商业测试工具并认为它可以自动执行您所有的功能测试,那么您可能从错误的地方开始。

You need to understand what problem you are trying to solve. What are you trying to automate? For example, if you have no test automation of any kind, and you start by buying an expensive commercial test tool thinking it will automate all your functional tests, you may be starting in the wrong place.

我们建议您从头开始。寻找你最大的收获。最划算的绝对是程序员可以做的单元测试。与其从测试金字塔的顶部开始,不如从底部开始,确保基础知识到位。你还需要考虑您需要自动化的不同类型的测试,以及何时需要准备好可用的工具。

We suggest you start at the beginning. Look for your biggest gain. The biggest bang for the buck is definitely the unit tests that the programmers can do. Instead of starting at the top of the test pyramid, you may want to start at the bottom, making sure that the basics are in place. You also need to consider the different types of tests you need to automate, and when you’ll need to have tools ready to use.

在本节中,我们假设您已经对第 1 象限的单元和组件测试进行了自动化,并希望在第 2 和第 3 象限中实现面向业务的测试自动化,或者使您的第 4 象限面向技术的测试对产品进行评估。我们将帮助您设计一个好的策略来构建您的自动化资源。

In this section, we assume you have automated Quadrant 1 unit and component tests in place, and are looking to automate your business-facing tests in Quadrants 2 and 3, or your Quadrant 4 technology-facing tests that critique the product. We’ll help you design a good strategy for building your automation resources.

想想你团队的技能和经验。谁需要自动化,为什么?你想达到什么目标?了解其中一些问题可能会影响您对工具的选择以及您付出的努力。本章末尾有一节是关于评估工具的。

Think about the skills and experience on your team. Who needs the automation, and why? What goals are you trying to achieve? Understanding some of these issues may affect your choice of tools and what effort you expend. There is a section on evaluating tools at the end of this chapter.

自动化很可怕,尤其是如果您是从头开始,那么我们从哪里开始呢?

Automation is scary, especially if you’re starting from scratch, so where do we begin?

哪里最痛?

Where Does It Hurt the Most?

要弄清楚下一步应该将自动化工作的重点放在何处,请询问您的团队,“最痛苦的地方是什么?” 或者,对于某些团队来说,“最无聊的地方是什么?” 你甚至可以部署代码来测试它吗?团队成员对更改代码是否有信心,或者他们是否缺乏自动化测试的任何安全网?也许您的团队成员更高级,掌握了 TDD,并且拥有全套单元测试。但是他们没有一个好的框架来指定面向业务的测试,或者不能完全掌握自动化测试的方法。也许您确实有一些 GUI 测试,但它们非常慢并且维护成本很高。

To figure out where to focus your automation efforts next, ask your team, “What’s the greatest area of pain?” or, for some teams, “What’s the greatest area of boredom?” Can you even get code deployed in order to test it? Do team members feel confident about changing the code, or do they lack any safety net of automated tests? Maybe your team members are more advanced, have mastered TDD, and have a full suite of unit tests. But they don’t have a good framework for specifying business-facing tests, or can’t quite get a handle on automating them. Perhaps you do have some GUI tests, but they’re extremely slow and are costing a lot to maintain.

危险:尝试手动测试所有内容

Peril: Trying to Test Everything Manually

如果您将所有时间都花在重新测试之前测试过的功能上,而不是接触新功能,并且需要添加越来越多的测试,那么您就会严重缺乏测试自动化。这种危险意味着测试人员没有时间参与设计和实现讨论,回归错误可能会悄悄潜入,测试无法跟上开发的步伐,并且测试人员会陷入困境。开发人员没有参与面向业务的测试,测试人员也没有时间找出更好的方法来解决测试问题。

If you’re spending all your time retesting features that you’ve tested before, not getting to new features, and needing to add more and more testing, you’re suffering from a severe lack of test automation. This peril means that testers don’t have time to participate in design and implementation discussions, regression bugs may creep in unnoticed, testing can’t keep up anymore with development, and testers get stuck in a rut. Developers aren’t getting involved in the business-facing testing, and testers don’t have time to figure out a better way to solve the testing problems.

正如我们在本章中所描述的,您的团队可以通过制定自动化策略来解决这个问题。团队开始设计可测试性并选择和实施适当的自动化工具。测试人员有机会发展他们的技术技能。

Your team can fix this by developing an automation strategy, as we describe in this chapter. The team starts designing for testability and chooses and implements appropriate automation tools. Testers get an opportunity to develop their technical skills.

无论哪里受到的伤害最大,那就是您开始自动化工作的地方。例如,如果您的团队甚至难以交付可部署的代码,您就需要实施自动化构建流程。没有什么比在等待一些代码测试时摆弄拇指更糟糕的了。

Wherever it hurts the most, that’s the place to start your automation efforts. For example, if your team is struggling to even deliver deployable code, you need to implement an automated build process. Nothing’s worse than twiddling your thumbs while you wait for some code to test.

但是,如果性能使您的组织处于危险之中,则性能测试必须是重中之重。这又回到了了解您要解决的问题上。风险分析是你的朋友。

But, if performance puts the existence of your organization in danger, performance testing has to be the top priority. It’s back to understanding what problem you are trying to solve. Risk analysis is your friend here.

第 18 章编码和测试”包含有关风险分析的简单方法的更多信息。

Chapter 18, “Coding and Testing,” has more information on a simple approach to risk analysis.

珍妮特的故事

Janet’s Story

我在一个遗留系统上工作,该系统试图解决一些质量问题并为我们的主要客户添加新功能。现有应用程序没有自动化单元或功能测试,但我们需要重构代码以解决质量问题。团队成员决定一次解决一个问题。当他们选择要重构的功能块时,程序员编写单元测试,确保它们通过,然后重写代码,直到测试再次通过。在重构结束时,他们有了可测试的、编写良好的代码和测试。测试人员同时编写更高级别的功能测试。一年之内,大部分质量低劣的遗留代码都被重写了,团队仅通过一次处理一个块就实现了良好的测试覆盖率。

I worked on a legacy system that was trying to address some quality issues as well as add new features for our main customer. There were no automated unit or functional tests for the existing application, but we needed to refactor the code to address the quality issues. The team members decided to tackle it one piece at a time. As they chose a chunk of functionality to refactor, the programmers wrote unit tests, made sure they passed, and then rewrote the code until the tests passed again. At the end of the refactoring, they had testable, well-written code and the tests to go with them. The testers wrote the higher-level functional tests at the same time. Within a year, most of the poor-quality legacy code had been rewritten, and the team had achieved good test coverage just by tackling one chunk at a time.

——珍妮特

—Janet

除非有其他良好的开发实践,否则测试自动化不会带来回报。运行一套健壮的单元测试的持续集成是实现其他测试自动化的第一步。为可维护性和良好设计而不断重构的代码将有助于提高自动化的投资回报率。没有良好的单元测试覆盖率,就不可能进行重构。这些开发实践也需要应用于自动化功能测试脚本。

Test automation won’t pay off unless other good development practices are in place. Continuous integration running a robust suite of unit tests is a first step toward automating other tests. Code that’s continually refactored for maintainability and good design will help increase the ROI on automation. Refactoring can’t happen without that good unit test coverage. These development practices also need to be applied to the automated functional test scripts.

多层方法

Multi-Layered Approach

虽然我们建议一次掌握一种工具,但不要对任何一种工具期望过高。为每个需要使用正确的工具。最适合单元测试的工具可能适合也可能不适合自动化功能测试。GUI、负载、性能和安全测试可能各自需要一个或多个不同的工具。

While we recommend mastering one tool at a time, don’t expect too much out of any one tool. Use the right tool for each need. The tool that works best for unit tests may or may not be appropriate to automate functional tests. GUI, load, performance, and security testing may each require a different tool or tools.

Mike Cohn 的测试金字塔概念(参见图 14-2)帮助我们的团队将他们的自动化工作放在他们最擅长的地方。我们希望最大化具有最佳投资回报率的测试。如果系统架构是为可测试性而设计的,那么测试自动化的成本就会更低,尤其是在单元级别。

Mike Cohn’s test pyramid concept (see Figure 14-2) has helped our teams put their automation efforts where they do the most good. We want to maximize the tests that have the best ROI. If the system architecture is designed for testability, test automation will be less expensive, especially at the unit level.

通过用户界面进行的测试通常具有最低的投资回报率,因为它们的维护成本很高,但我们确实需要它们。它们构成了我们金字塔的一小部分 我们可能会选择将其中一些测试自动化,但大多数 GUI 测试都是用业务术语定义的,可能最好保留为人机交互测试(即手动测试)。

Tests that go through the user interface usually have the lowest ROI, because they’re expensive to maintain, but we do need them. They make up the small tip of our pyramid We may choose to automate some of these tests, but the majority of GUI tests are defined in business terms and probably are best left as human interaction tests (i.e., manual tests).

中间层代表直接与生产代码一起工作的功能测试,中间没有 GUI 或其他层。虽然它们的自动化成本不如单元级测试那么低,而且提供反馈的速度也稍慢一些,但正确的工具可以让它们获得良好的投资回报率。这些测试可以用业务专家理解的语言编写,这一事实增加了它们的价值。

The middle layer represents the functional tests that work directly with production code, without a GUI or other layer in between. While they’re not as inexpensive to automate as unit-level tests, and provide feedback a bit more slowly, the right tools allow them to have a good ROI. The fact that these tests can be written in a language the business experts understand adds to their value.

应用程序中有许多不同的层可以独立测试。在他的书xUnit 测试模式[2007] 中,Gerard Meszaros 将此称为层测试模式。他告诫说,当尝试单独测试应用程序的所有层时,我们仍然必须验证这些层是否正确连接,这可能需要至少通过表示层对业务逻辑进行一次测试。

There are many different layers in the application that can be tested independently. In his book xUnit Test Patterns [2007], Gerard Meszaros refers to this as the Layer Test pattern. He cautions that when trying to test all of the layers of the application separately, we still have to verify that the layers are hooked up correctly, and this may require at least one test of the business logic through the presentation layer.

丽莎的故事

Lisa’s Story

当我的团队一步步构建我们的自动化框架时,我们收集了一系列工具。在使用 Ant 和 CruiseControl 实现持续构建框架后,我们掌握了用于单元测试的 JUnit。我们知道单元测试自动化是最快和最便宜的自动化方式,并为程序员提供最快的反馈。

As my team built our automation framework one step at a time, we gathered an arsenal of tools. After implementing a continuous build framework with Ant and CruiseControl, we mastered JUnit for unit testing. We knew that unit test automation is the quickest and cheapest way to automate, and provides the fastest feedback to the programmers.

我们的遗留系统没有自动化测试,因此我们使用 Canoo WebTest 构建了一个 GUI 回归测试套件。这提供了良好的回报,因为 WebTest 脚本是指定的,而不是编程的。它们编写起来很快并且易于维护。

Our legacy system had no automated tests, so we built a GUI regression test suite with Canoo WebTest. This provided good payback because the WebTest scripts were specified, not programmed. They were quick to write and easy to maintain.

在 JUnit 和 WebTest 到位后,我们对 FitNesse 进行了试验,发现它在 GUI 背后的功能测试方面效果很好。我们发现使用 FitNesse 进行自动化的速度相对较快。尽管 FitNesse 测试的生产和维护成本明显高于单元测试,但它们在推动开发和促进客户、程序员和测试人员之间的协作方面的价值保持了较高的投资回报率。

After JUnit and WebTest were in place, we experimented with FitNesse and found it worked well for functional testing behind the GUI. We found automating with FitNesse to go relatively quickly. Although FitNesse tests are significantly more expensive to produce and maintain than unit tests, their value in driving development and promoting collaboration among customers, programmers, and testers kept the ROI high.

所有这些工具都易于学习、实施并与构建过程集成,并提供有关回归问题的持续反馈。当我们决定我们的测试自动化策略时,它们是重要的考虑因素。

All of these tools were easy to learn, implement, and integrate with the build process, and provided continual feedback about our regression issues. They were important considerations when we were deciding on our test automation strategy.

——丽莎

—Lisa

在评估自动化工作的回报时,请考虑不太具体的方面,例如该工具是否促进了团队之间的协作技术和客户团队。编写测试的一个主要原因是帮助指导开发。如果编写自动化验收测试的过程导致对业务需求的透彻理解,那将是巨大的回报,即使测试以后再也找不到一个单一的回归错误。

When evaluating the payback of your automation efforts, consider less tangible aspects such as whether the tool promoted collaboration between the technical and customer teams. A primary reason to write tests is to help guide development. If the process of writing your automated acceptance tests results in a thorough understanding of business requirements, that’s plenty of payback, even if the tests never find a single regression bug later on.

考虑测试设计和维护

Think about Test Design and Maintenance

想一想您一生中编写的所有手动测试脚本。难道你不希望所有这些都自动化吗?你的生活不会轻松很多吗?我们认为所有脚本测试都应该自动化。让我们开始转换那些手动脚本测试。

Think about all of the manual test scripts you’ve written in your life. Don’t you just wish those all would have been automated? Wouldn’t your life have been a lot easier? We believe that all scripted tests should be automated. Let’s get started converting those manual scripted tests.

开始后,自动化测试会非常容易。例如,当您有一个工作的 FitNesse 夹具时,添加更多测试用例几乎不需要付出任何努力。当您有很多不同的排列要测试时,这很棒。与手动完成所有测试相比,您可能会测试更多的条件。当 Lisa 的团队成员重写他们退休计划的贷款系统时,他们可以通过 FitNesse 测试来测试数百种不同的贷款支付处理可能性。如果同一天处理三笔贷款付款会怎样?如果有人三个月没有支付任何款项,然后又支付了一大笔款项,那么利息的计算和应用是否正确?很容易编写自动化测试来找出答案。

After you get started, it can be quite easy to automate tests. For example, when you have a working FitNesse fixture, adding more test cases requires little effort. This is great when you have a lot of different permutations to test. You’ll probably test more conditions than you would if all your testing was done manually. When Lisa’s team members rewrote their retirement plan’s loan system, they could test hundreds of different possibilities for loan payment processing via FitNesse tests. What happens when three loan payments are processed on the same day? If someone doesn’t make any payments for three months, and then sends in a large payment, is the interest calculated and applied correctly? It was easy to write automated tests to find out.

这是一个很大的优势,但也有不利的一面。现在团队有几十个,甚至上百个测试用例需要维护。如果一年后计算贷款利息金额的规则发生变化怎么办?这可能需要更新每个测试。如果您的测试工具不能轻易适应对现有测试的更改,那么您的大型自动化测试套件可能会变得令人头疼。

That’s a great advantage, but it has a down side. Now the team has dozens, or even hundreds, of test cases to maintain. What if the rules about calculating the amount of interest for a loan payment change a year from now? This could require updating every test. If your test tool doesn’t easily accommodate making changes to existing tests, your big suite of automated tests can turn into a headache.

端到端测试特别难以自动化,因为随着业务规则的变化,它们最有可能需要维护。我们如何平衡自动化的需求与成本?

End-to-end tests are particularly tricky to automate because they have the most potential to need maintenance as business rules change. How do we balance the need for automation with the cost?

测试设计

请记住从您正在测试的功能的薄片或钢丝开始。就像程序员接近编码一样接近自动化。让一小部分钢丝工作,然后继续下一个。在你覆盖了整个薄片之后,回去把它充实起来。

Remember to start with the thin slice or steel thread of the feature you’re testing. Approach automation just as programmers approach coding. Get one small unit of the steel thread working, and then move on to the next. After you’ve covered the whole thin slice, go back and flesh it out.

第 8 章,“支持团队的面向业务的测试”,详细解释了薄片。

Chapter 8, “Business-Facing Tests that Support the Team,” explains more about thin slices.

仔细选择您的测试模式。自动化您需要的所有测试用例,但仅此而已,并在您可以的最低级别自动化它们。将每个测试用例的范围限制为一个测试条件或一个业务规则。了解测试的目的。避免测试之间的依赖关系,因为它们会迅速增加复杂性和维护费用。

Choose your test pattern thoughtfully. Automate all of the test cases you need, but no more, and automate them at the lowest level that you can. Limit the scope of each test case to one test condition or one business rule. Understand the purpose of the test. Avoid dependencies between tests, because they quickly increase complexity and maintenance expense.

有关有效测试设计的更多信息,请参阅第 9 章支持团队的面向业务的测试工具包”。

See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for more information about effective test design.

考虑选项

正如我们之前提到的,自动化测试的级别越低,投资回报率就越高。尽可能将测试自动化推向金字塔的下端。如果您的单元测试和代码集成测试覆盖率很高,则不需要自动化那么多的功能测试。有了较低级别的可靠覆盖,手动进行端到端测试来验证系统的行为可能就足够了。使用风险分析来帮助您做出决定。

As we’ve mentioned before, the lower the level at which you automate a test, the better the ROI. Push test automation as far down the pyramid as you can. If you have good coverage in your unit and code integration tests, you don’t need to automate as many functional tests. With solid coverage at the lower levels, it might be enough to do end-to-end tests manually to verify the system’s behavior. Use risk analysis to help you decide.

用户界面

用户界面确实需要测试。在某些情况下,GUI 级别的测试自动化至关重要。也许您的团队正在使用第三方 GUI 控件,而您不确定它们的行为方式。如果您的风险和 ROI 分析支持 GUI 级别的大量自动化,请进行投资。

The user interface does need to be tested. In some situations, test automation at the GUI level is critical. Perhaps your team is using third-party GUI controls, and you aren’t sure how they will behave. If your risk and ROI analysis supports a lot of automation at the GUI level, make the investment.

如果您在更高级别进行自动化,请不要过火并使系统中的每条可能路径都自动化。您不必将在开发阶段创建的每个自动化测试都保留在回归套件中;考虑构建时间和发现缺陷的机会之间的权衡。集中精力覆盖单元、代码集成和功能级别的代码中的每条重要路径。你会得到更好的回报。

If you do automate at the higher levels, don’t go overboard and automate every possible path through the system. You don’t have to keep every automated test created during the development phase in the regression suite; consider tradeoffs of build time and the chance of finding defects. Focus your efforts on covering every important path through the code at the unit, code integration, and functional levels. You’ll get a much better payback.

冲帐

取得平衡不是敏捷原则,它只是常识。您现在需要一个足够好的解决方案,但它不一定是完美的。该工具是否提供您现在需要的结果?它是否为将其用于自动化所需的资源提供了足够的回报?如果是这样,请继续使用它,并在以后安排时间寻找替代方案。随着时间的推移,您可以改进您的自动化框架。最重要的因素是您的自动化工具现在是否适合您的特定情况。

Striking a balance isn’t an agile principle, it’s just common sense. You need a good-enough solution right now, but it doesn’t have to be perfect. Does the tool provide the results you need right now? Does it provide an adequate return on the resources needed to use it for automation? If so, go ahead and use it, and budget time later to look for alternatives. You can improve your automation framework over time. The most important factor is whether your automation tools fit your particular situation right now.

不要反过来想,“好吧,我们可以用这个记录工具生成一堆脚本,立即完成测试,然后重构脚本以使其可维护。” 虽然您不需要一直寻找完美的自动化解决方案,但您确实需要一个不增加您团队的技术债务。在“它找到我们需要了解的错误并且维护成本不高”和“这是我们能找到的最优雅、最酷的解决方案”之间找到平衡。

Don’t slide the other way, and think, “OK, we can generate a bunch of scripts with this record tool, get our immediate testing done, and refactor the scripts later to make them maintainable.” While you don’t need to keep searching for the perfectly ideal automation solution, you do need a solution that doesn’t add to your team’s technical debt. Find a balance between “It finds the bugs we need to know about and doesn’t cost too much to maintain” and “This is the most elegant and cool solution we can find.”

选择合适的工具

Choosing the Right Tools

很酷的是,我们有这么多工具可以帮助我们解决自动化问题。不要追求比你需要的更复杂的东西。Lisa 的同事发现,从数据库中检索数据并独立于系统执行计算的电子表格是一种强大的工具,既可用于推动开发,也可用于验证应用程序的计算。

It’s cool that we have so many tools available to help us solve our automation problems. Don’t go for more sophistication than you need. Lisa’s coworkers have found that a spreadsheet that retrieves data from the database and performs calculations independently of the system is a powerful tool, both for driving development and for verifying the application’s calculations.

我们通常尽量减少 GUI 层的测试自动化,但在某些情况下,更多的 GUI 自动化是合适的。如果用户在 X 处进行了更改,还有什么变化?有些问题仅在 GUI 级别表现出来。当退休计划参与者要求从他们的账户中分配资金时,Lisa 测试了一个解决后端问题的错误修复。更改被单元测试包围,但它是一个 GUI 回归测试,当分发表单无法根据请求弹出时失败。没有人预料到后端更改会影响 GUI,因此他们可能不会费心手动测试它。这也是您需要 GUI 回归测试的原因。

We usually minimize test automation at the GUI layer, but there are situations where more GUI automation is appropriate. If the user makes a change at X, what else changes? Some problems only manifest themselves at the GUI level. Lisa tested a bug fix that addressed a back-end problem when retirement plan participants requested a distribution of money from their accounts. The change was surrounded by unit tests, but it was a GUI regression test that failed when the distribution form failed to pop up upon request. Nobody anticipated that a back-end change could affect the GUI, so they probably wouldn’t have bothered to test it manually. That’s why you need GUI regression tests, too.

我们已经讨论了录制/回放工具的一些缺点,但它们适用于正确的情况。您可能有充分的理由使用记录/回放工具:也许您的遗留代码已经在该工具中创建了一套自动化测试,您的团队在该工具方面拥有丰富的专业知识,或者您的管理层希望您将其用于不管什么原因。您可以使用录制的脚本作为起点,然后将脚本分解为模块,在适当的地方用参数替换硬编码数据,并使用模块作为构建块来组装测试。即使您没有太多编程经验,也不难识别模块中应包含的脚本块。例如,登录是一个显而易见的选择。

We’ve talked about some disadvantages of record/playback tools, but they’re appropriate in the right situation. You may be using a record/playback tool for a good reason: Maybe your legacy code already has a suite of automated tests created in that tool, your team has a lot of expertise in the tool, or your management wants you to use it for whatever reason. You can use recorded scripts as a starting point, then break the scripts into modules, replace hard-coded data with parameters where appropriate, and assemble tests using the modules as building blocks. Even if you don’t have much programming experience, it’s not hard to identify the blocks of script that should be in a module. Login, for example, is an obvious choice.

记录/回放也可能适用于遗留系统,这些遗留系统的设计方式使得单元测试变得困难,并且从头开始手工编写脚本测试的成本太高。可以在应用程序中构建记录和回放功能,甚至是遗留应用程序。通过正确的设计,并为记录的交互使用一些人类可读的格式,甚至可以在构建代码之前构建回放测试。

Record/playback may also be appropriate for legacy systems that are designed in such a way that makes unit testing difficult and hand-scripting tests from scratch too costly. It’s possible to build a record and playback capability into the application, even a legacy application. With the right design, and the use of some human-readable format for the recorded interaction, it’s even possible to build playback tests before the code is built.

一些敏捷团队从商业或开源测试工具中获得价值,而其他团队则更喜欢完全定制的方法。许多测试人员发现使用 Ruby 或 shell 等脚本语言编写简单的脚本来自动执行平凡但必要的任务、生成测试数据或驱动其他工具很有价值。Everyday Scripting with Ruby for Teams, Testers, and You等书籍给出了这种方法的路线图。如果您是一名没有强大编程背景的测试人员,我们鼓励您拿起一本书、查找在线教程或参加有关脚本语言的课程,看看编写有用的脚本是多么容易。

Some agile teams get value from commercial or open source test tools, while others prefer a completely customized approach. Many testers find value writing simple scripts in a scripting language such as Ruby, or a shell, to automate mundane but necessary tasks, generate test data, or drive other tools. Books such as Everyday Scripting with Ruby for Teams, Testers, and You give a roadmap for this approach. If you’re a tester without a strong programming background, we encourage you to pick up a book, find an online tutorial, or take a class on a scripting language, and see how easy it can be to write useful scripts.

在第 9 章支持团队的面向业务测试的工具包”中查看面向业务测试的特定工具的更多示例。

See more examples of specific tools for business-facing tests in Chapter 9, “Toolkit for Business-Facing Tests that Support the Team.”

我们想告诉您的是,您可以使用许多不同的工具。看看你正试图解决的问题,然后作为一个团队来决定解决它的最简单和最有效的方法。每隔一段时间,退一步看看工具你正在使用。团队中的每个人都对他们满意吗?您是否因为没有合适的工具而遗漏了问题?预算时间探索新工具,看看它们是否可以填补空白或更换没有回报的工具。

What we’re trying to tell you is that you can use many different tools. Look at the problem you are trying to solve and decide as a team the easiest and most effective way to solve it. Every so often, step back and take a look at the tools you’re using. Is everyone on the team happy with them? Are you missing problems because you don’t have the right tools? Budget time to explore new tools and see if they might fill gaps or replace a tool that isn’t paying off.

如果您的团队是敏捷开发的新手,或者正在处理一个全新的项目,您可能会在早期迭代期间面临选择工具和设置测试环境的问题,同时您还可能处理高风险的故事。如果您仍在创建测试基础设施,就不要指望能够提供太多的商业价值。计划大量时间来评估工具、设置构建过程以及尝试不同的测试方法。

If your team is new to agile development, or working on a brand-new project, you might be faced with choosing tools and setting up test environments during the early iterations, when you also might be working on high-risk stories. Don’t expect to be able to deliver much business value if you’re still creating your test infrastructure. Plan in lots of time for evaluating tools, setting up build processes, and experimenting with different test approaches.

将敏捷原则应用于测试自动化

Applying Agile Principles to Test Automation

每个团队、每个项目和每个组织都有独特的情况和独特的自动化挑战。每个人都有自己的文化、历史、资源、业务压力、产品和经验。无论您的团队的情况如何,您都可以使用第 2 章中讨论的敏捷原则和价值观来帮助您找到解决方案。勇气、反馈、简单、沟通、持续改进和响应变化等概念不仅仅是敏捷的想法——它们是所有成功团队共有的品质。

Every team, every project, and every organization has a unique situation with unique automation challenges. Each has its own culture, history, resources, business pressures, products, and experience. No matter what your team’s situation, you can use the agile principles and values discussed in Chapter 2 to help you find solutions. Concepts such as courage, feedback, simplicity, communication, continuous improvement, and responding to change aren’t just agile ideas—they’re qualities that are common to all successful teams.

把事情简单化

Keep It Simple

“做可能有效的最简单的事情”的敏捷格言适用于测试和代码。保持测试设计简单,范围最小,并使用最简单的工具来完成这项工作。

The agile maxim of “do the simplest thing that could possibly work” applies to tests as well as code. Keep the test design simple, keep the scope minimal, and use the simplest tool that will do the job.

简单是敏捷的核心价值,这是有充分理由的。最好的起点是您能想到的最简单的方法。然而,做最简单的事并不意味着做最容易的事。它涉及真正思考您现在需要什么,并采取一些小步骤来实现目标。通过保持简单,如果您确实做出了错误的选择,您就不会在意识到自己的方法错误之前偏离轨道太远。

Simplicity is a core agile value for a good reason. The best place to start is the simplest approach you can think of. However, doing the simplest thing doesn’t mean doing the easiest thing. It involves really thinking about what you need now and taking baby steps to get there. By keeping things simple, if you do make a bad choice, you won’t go too far off track before realizing the error of your ways.

很容易参与一项任务,并从基础知识中溜走,进入一些有趣的挑战中。在执行之前权衡每项自动化任务的投资回报率。自动化很有趣(当您克服入门的可怕部分时)。尝试一些困难的事情是很诱人的,因为你可以。与敏捷开发项目中测试的所有其他方面一样,跟上步伐的唯一方法是只做所需的最少工作。

It’s easy to get involved in a task and slip away from the basics into some intriguing challenge. Weigh the ROI of every automation task before you do it. Automation is fun (when you get past the scary part of getting started). It’s tempting to try something difficult just because you can. Like all other aspects of testing in an agile development project, the only way to keep up is to do only the minimum required.

使用您可以使用的最简单的工具。记住测试金字塔。如果面向客户的测试最容易在单元级别实现自动化,那就去做吧那里。Lisa 有时在 FitNesse 中编写测试用例,只是为了了解程序员可以像 JUnit 测试一样更快地自动化它们。相反,有时程序员将 FitNesse 用于 TDD 而不是 JUnit,因为他们编写的代码适合以其中一种 FitNesse 夹具格式进行测试。

Use the simplest tool you can get away with. Remember the test pyramid. If a customer-facing test can be most easily automated at the unit level, do it there. Lisa sometimes writes test cases in FitNesse, only to learn the programmers can automate them much faster as JUnit tests. Conversely, sometimes the programmers use FitNesse for TDD instead of JUnit, because the code they’re writing lends itself to testing in one of the FitNesse fixture formats.

迭代反馈

Iterative Feedback

短迭代使我们能够试验各种自动化方法、评估结果并根据需要尽快改变路线。致力于自动化工作,例如在内部开发测试框架或实施开源工具至少进行几次迭代。每次迭代后,查看哪些有效,哪些无效。想出克服问题的想法,并在下一次迭代中尝试这些想法。如果它不是正确的解决方案,请尝试其他方法进行几次迭代。不要陷入泥潭,你已经将如此多的资源投入到一个工具中,并且有如此多的测试使用它,以至于你觉得你无法切换工具。在许多开源和商业工具之间,加上程序员编写自己开发的测试工具的能力,没有理由满足于低于最佳的工具。

Short iterations allow us to experiment with various automation approaches, evaluate results, and change course as quickly as needed. Commit to an automation effort, such as developing a test framework in-house or implementing an open source tool for at least a couple of iterations. After each iteration, look at what’s working and what’s not working. Think of ideas to overcome problems, and try those in the next iteration. If it’s not the right solution, try something else for a few iterations. Don’t get sucked into a quagmire where you’ve put so many resources into a tool, and have so many tests that use it, that you feel you can’t switch tools. Between the many open source and commercial tools, plus programmers’ ability to write home-grown test tools, there’s no reason to settle for less than the optimum tool.

丽莎的故事

Lisa’s Story

我的一个早期 XP 团队努力寻找一种好方法来自动化基于 Java 的 Web 应用程序的面向客户的验收测试。这是在敏捷团队的工具选择要少得多的时候。首先,我们尝试了一个模拟浏览器的开源工具,但它缺少我们需要的功能。它只是不够坚固。我们在下一次回顾中讨论了这个问题。

One of my early XP teams struggled to find a good way to automate customer-facing acceptance tests for a Java-based web application. This was back when there were far fewer tool options for agile teams. First, we tried an open source tool that simulated a browser, but it lacked the features we required. It just wasn’t quite robust enough. We discussed this at the next retrospective.

我们决定在接下来的两次迭代中尝试使用单元测试工具在 GUI 后面进行测试。通过进行两次迭代,我们觉得我们给了自己足够的时间来很好地尝试该工具,但又没有太多时间,以至于如果它不是正确的解决方案,我们就会投入太多。客户发现单元测试难以阅读,并且 GUI 中存在我们无法使用此工具测试的逻辑。

We decided to try using the unit testing tool for testing behind the GUI for the next two iterations. By committing to two iterations, we felt we were giving ourselves enough time to give the tool a good try, but not so much time that we would have too much invested if it weren’t the right solution. The customers found the unit tests hard to read, and there was logic in the GUI we couldn’t test with this tool.

在我们回顾期间的另一次讨论之后,我们承诺使用我在以前的项目中广泛使用的供应商 GUI 测试工具进行两次迭代。Java 程序员发现它进展缓慢,因为该工具使用专有脚本语言,但它运行良好,足以完成所需的最低限度自动化。两次迭代后,我们认为它并不理想,但当时并没有太多其他选择,它是我们拥有的最好的一个。

After another discussion during our retrospective, we then committed to two iterations of using a vendor GUI test tool I had used extensively on previous projects. The Java programmers found it slow going because the tool used a proprietary scripting language, but it worked well enough to do the minimum automation needed. After two iterations, we decided that it wasn’t ideal, but at the time there weren’t a lot of other options, and it was the best one we had.

事后看来,我们应该继续寻找更好的选择。也许我们可以开发自己的测试工具。我们能够使用供应商工具自动化大约 60% 的单元级别以上的回归测试,这在当时看起来很棒。如果我们给自己多一点压力,我们可能会做得更好。

In hindsight, we should have kept looking for a better option. Perhaps we could have developed our own test harness. We were able to automate about 60% of the regression tests above the unit level using the vendor tool, which seemed great at the time. If we had pushed ourselves a little more, we might have done a lot better.

——丽莎

—Lisa

使用迭代对你有利。它们促进了循序渐进的方法。如果您的想法是一个失败的想法,您很快就会知道并有机会尝试不同的想法。不要害怕继续寻找,但如果您尝试的解决方案表现良好,也不要继续寻找完美的解决方案。

Use iterations to your advantage. They facilitate a step-wise approach. If your idea’s a dud, you’ll know quickly and have a chance to try a different one. Don’t be afraid to keep looking, but don’t keep looking for the perfect solution if one you try performs adequately.

全团队方法

Whole-Team Approach

没有自动化,敏捷开发就无法工作。幸运的是,我们在第 1 章中探讨的全团队方法意味着更广泛的技能和资源可用于找到和实施有用的自动化策略。作为一个团队解决问题意味着更有可能将代码设计为可测试性。程序员、测试人员和其他团队成员将协作实现测试自动化,为这项工作带来多种观点和技能组合。

Agile development can’t work without automation. Fortunately, the whole-team approach, which we explored in Chapter 1, means that a wider range of skills and resources are available to find and implement a useful automation strategy. Attacking the problem as a team means it’s more likely that code will be designed for testability. Programmers, testers, and other team members will collaborate to automate tests, bringing multiple viewpoints and skill sets to the effort.

整个团队的方法有助于克服恐惧障碍。自动化任务一开始可能会让人不知所措。知道还有其他具有不同技能和经验的人可以帮助我们,这给了我们勇气。能够寻求和获得帮助让我们相信我们可以通过自动化测试实现足够的覆盖率。

The whole-team approach helps overcome the fear barrier. Automation tasks can be overwhelming to start with. Knowing there are other people with different skills and experience to help gives us courage. Being able to ask for and receive help gives us confidence that we can achieve adequate coverage with our automated tests.

丽莎的故事

Lisa’s Story

我目前的团队承诺在所有有意义的级别上进行自动化回归测试。以下是我寻求帮助以成功实现自动化的一些示例。

My current team made a commitment to automating regression tests at all levels where it made sense. Here are some examples of where I’ve asked for help to succeed with automation.

早期,当我们根本没有自动化测试并且开发人员试图掌握测试驱动开发时,我们选择了 Canoo WebTest 进行 GUI 冒烟测试。我需要一些帮助来了解如何配置 WebTest 以在我们的环境中运行,并且我需要很多帮助来运行自动构建过程中的测试。我请我们的系统管理员(也是程序员之一)帮忙。我们很快就在构建中运行了一套测试。

Early on, when we had no automated tests at all and the developers were trying to master test-driven development, we settled on Canoo WebTest for the GUI smoke tests. I needed a bit of help understanding how to configure WebTest to run in our environment, and I needed a lot of help to run the tests from the automated build process. I asked our system administrator (who was also one of the programmers) to help. We quickly got a suite of tests running in the build.

后来真的很想试试FitNesse做GUI背后的功能测试。当程序员仍在使用自动化单元测试时,我必须耐心等待。该团队同意试用该工具,但很难找到时间开始使用它。我选择了一个似乎适合 FitNesse 测试的故事,并询问编写该故事的程序员是否可以与他配对来尝试一些 FitNesse 测试。他同意了,我们在 FitNesse 中进行了一些自动化测试。程序员觉得这很容易,也很值得,并向团队的其他成员做了一个很好的报告。

Later, I really wanted to try FitNesse for functional testing behind the GUI. I had to be patient while the programmers were still getting traction with the automated unit tests. The team agreed to try the tool, but it was hard to find time to start using it. I picked a story that seemed suited to FitNesse tests, and asked the programmer working on the story if I could pair with him to try some FitNesse tests. He agreed, and we got some tests automated in FitNesse. The programmer found it easy and worthwhile, and gave a good report to the rest of the team.

之后,不难接近每个程序员,建议为他正在处理的故事编写 FitNesse 测试,并让他看到结果。FitNesse 测试发现了程序员没有想到的测试用例,他们立即看到了好处。当团队中的每个人都有一些使用该工具的经验时,他们不仅乐于自动化测试,而且开始以一种可以更轻松地编写 FitNesse 夹具的方式设计代码。

After that, it wasn’t hard to approach each programmer, suggest writing FitNesse tests for the story he was working on, and let him see the results. The FitNesse tests found test cases the programmer hadn’t thought of, and they saw the benefit right away. When everyone on the team had some experience with the tool, they were not only happy to automate the tests, but started designing code in a way that would make writing FitNesse fixtures easier.

当设计了我们大部分 Watir 测试套件的 Ruby 专家离开公司时,我非常担心维护我们庞大的测试套件以及能够编写新的测试套件。我的 Ruby 专业知识不如他(另外,我们只剩下一名测试人员,所以时间是个问题)。团队中的每个程序员都出去买了一本关于 Ruby 的书,并在我更新脚本以在代码更改时工作时遇到问题时提供帮助。当我没有时间完成这项任务时,一位程序员甚至编写了一个新脚本来测试一个新故事。当我们聘请了一名新测试员时,他和我能够处理 Watir 脚本的维护和提供,因此程序员不再需要承担这些任务。

When our Ruby expert, who designed most of our Watir test suite, left the company, I was quite concerned about maintaining our huge suite of tests as well as being able to code new ones. My Ruby expertise was not as good as his (plus, we were down to just one tester, so time was an issue). Every programmer on the team went out, bought a book on Ruby, and helped when I had problems updating scripts to work when the code changed. One programmer even wrote a new script to test a new story when I didn’t have time for that task. When we hired a new tester, he and I were able to handle the care and feeding of the Watir scripts, so the programmers no longer needed to take on those tasks.

我知道我可以向队友寻求自动化问题的帮助,并且整个团队都将自动化视为优先事项,因此程序员在设计代码时总是考虑可测试性。这是整个团队工作方法的一个例子。

I know I can ask teammates for help with automation issues, and the entire team sees automation as a priority, so the programmers always think about testability when designing the code. This is an example of the whole-team approach at work.

——丽莎

—Lisa

专门的面向技术的测试,例如安全或负载测试,可能需要从团队外部引入专家。一些公司拥有专家团队,可以作为产品团队的共享资源。即使在利用这些资源的同时,敏捷团队仍应负责确保完成所有类型的测试。他们还可能会惊讶地发现,如果团队成员采取创造性的方法,他们可能具备所需的技能。

Specialized technology-facing tests such as security or load testing might require bringing in experts from outside the team. Some companies have specialist teams that are available as shared resources to product teams. Even while taking advantage of these resources, agile teams should still take responsibility for making sure all types of testing are done. They may also be surprised to find that team members may have the skills needed if they take a creative approach.

第 11 章,“使用面向技术的测试来评价产品”,讨论了诸如此类的面向技术的测试以及处理它们的不同方法。

Chapter 11, “Critiquing the Product Using Technology-Facing Tests,” talks about technology-facing tests such as these and different approaches to handling them.

一些组织有独立的测试团队进行开发后测试。他们可能正在测试以确保软件与其他系统集成,或者进行其他专门测试,例如大规模性能测试。开发团队应该与这些其他团队密切合作,利用所有测试工作的反馈来改进代码设计并促进自动化。

Some organizations have independent test teams that do post-development testing. They may be testing to ensure the software integrates with other systems, or conducting other specialized testing such as large-scale performance testing. Development teams should work closely with these other teams, using feedback from all testing efforts to improve code design and facilitate automation.

花时间做正确的事

Taking the Time to Do It Right

解决问题和实施好的解决方案需要时间。我们必须帮助我们的管理层了解,如果没有足够的时间以正确的方式做事,我们的技术债务将会增加,我们的速度将会减慢。以“正确”的方式实施解决方案需要时间,但从长远来看会节省时间。考虑一下集思广益、解决方案、正式培训和在职学习所需的时间。

Solving problems and implementing good solutions takes time. We must help our management understand that without enough time to do things the right way, our technical debt will grow, and our velocity will slow. Implementing solutions the “right” way takes time up front but will save time in the long term. Consider the time it takes for brainstorming ideas, solutions, formal training, and for on-the-job learning.

可以理解,您组织的管理层对尽快产生结果很感兴趣。如果管理层不愿意给团队时间来实施自动化,请清楚地解释权衡取舍。在没有自动回归测试的情况下在短期内交付一些功能以确保它们继续工作将会付出巨大的代价。随着您的团队积累技术债务,您将无法满足业务价值管理需求。努力达成妥协。例如,缩小功能范围但保留基本价值,并使用自动化来交付和维护更好的产品。

Your organization’s management is understandably interested in producing results as quickly as possible. If management is reluctant to give the team time to implement automation, explain the trade-offs clearly. Delivering some features in the short term without automated regression tests to make sure they keep working will have a big cost down the line. As your team accumulates technical debt, you’ll be less able to deliver the business value management needs. Work toward a compromise. For example, cut the scope of a feature but keep the essential value, and use automation to deliver and maintain a better product.

我们总是有最后期限,我们总是感到时间紧迫。回到我们一贯的方式做事的诱惑,比如手动执行回归测试并希望获得最好的结果,即使我们知道这是行不通的,但这种诱惑总是存在的。永远没有足够的时间回去解决问题。在您的下一次计划会议期间,安排时间在您的自动化工作上取得有意义的进展。

We always have deadlines, and we always feel pressed for time. The temptation to just go back to doing things the way we always have, like executing regression tests manually and hoping for the best, is always there even though we know that doesn’t work. There is never enough time to go back and fix things. During your next planning meeting, budget time to make meaningful progress on your automation efforts.

丽莎的故事

Lisa’s Story

我们的团队专注于花时间进行良好的设计、一套强大的自动化测试以及充足的时间进行探索性测试。质量,而不是速度,一直是我们的目标。我们的生产问题要花很多钱才能解决,所以整个公司都在努力花时间来预防这些问题。有时我们没有选择正确的设计,当我们意识到它时,我们并不害怕撕掉它并更换它。

Our team focuses on taking time for good design, a strong set of automated tests, and ample time for exploratory testing. Quality, not speed, has always been our goal. Our production problems cost a lot to fix, so the whole company is on board to take the time to prevent them. Sometimes we don’t pick the right design, and we aren’t afraid to rip it out and replace it when we realize it.

自然会有业务权衡,业务决定是否继续处理已知风险。我们努力清楚地解释所有风险并举例说明潜在情况。

Naturally there are business tradeoffs, and the business decides whether to proceed with known risks. We work to explain all of the risks clearly and give examples of potential scenarios.

这里有几个最近花时间把事情做好的例子。我们启动了一个主题,对退休计划的账户报表进行重大更改。其中一位程序员 Vince Palumbo 承担了收集用于语句的额外数据的任务。他决定为数据收集功能编写健壮的单元测试,即使这意味着故事必须继续到下一次迭代。编写单元测试需要花费大量的时间和精力,即使有测试,代码也极其复杂且难以实现。几次迭代后,另一位程序员 Nanda Lankalapalli 拿起了另一个与数据收集相关的故事,并惊喜地发现了新的单元测试。他能够快速做出改变,

Here are a couple of recent examples of taking the time to do things right. We started a theme to make major changes to the account statements for the retirement plans. One of the programmers, Vince Palumbo, took on the task of collecting additional data to be used for the statements. He decided to write robust unit tests for the data collection functionality, even though this meant the story would have to continue on to the next iteration. Writing the unit tests took a great deal of time and effort, and even with the tests, the code was extremely complex and difficult to do. A couple of iterations later, another programmer, Nanda Lankalapalli, picked up another story related to the data collection and was pleasantly surprised to find new unit tests. He was able to make his changes quickly, and the testing effort was greatly reduced because the unit tests were in place.

后来,我们发现我们错过了一个边缘案例,其中一些账户价值变化的计算是不正确的。自动化单元测试和大量探索性测试的结合不足以捕获所有场景。尽管如此,进行测试意味着 Vince 可以先测试再写出修正后的代码,并对代码现在的正确性更有信心。

Later, we found we had missed an edge case where some calculations for the change in account value were incorrect. The combination of automated unit tests and a great deal of exploratory testing were not enough to catch all of the scenarios. Still, having the tests meant Vince could write his corrected code test-first and feel more confident that the code was now correct.

最近的另一个例子涉及到支票的处理。该公司希望将两步流程缩短为一步,这意味着资金将比当时可能提前两天投资到退休计划账户。现有流程都是用遗留代码编写的,没有进行单元测试。我们讨论了是否要在新架构中重写处理。我们的产品负责人担心这可能需要多少时间。我们觉得更改现有代码与完全重写它所花的时间一样长,因为旧代码难以理解并且根本没有单元测试。我们决定重写,这不仅降低了这个关键功能出现问题的风险,而且让我们有机会以很少的额外成本提供一些额外的功能。到目前为止,这种策略已被证明是值得的。

Another recent example concerned processing of incoming checks. The business wanted to shorten the two-step process to one step, which meant the money would be invested in the retirement plan accounts two days earlier than was then possible. The existing process was all written in legacy code, without unit tests. We discussed whether to rewrite the processing in the new architecture. Our product owner was concerned about the amount of time this might take. We felt it would take just as long to change the existing code as to completely rewrite it, because the old code was difficult to understand and had no unit tests at all. We decided on the rewrite, which not only reduced the risk of problems in this critical functionality but also gave us the opportunity to provide a couple of extra features at little extra cost. So far, this strategy has proven worthwhile.

——丽莎

—Lisa

让自己成功。以可持续的速度工作。边走边花时间重构,否则最终会搞得一团糟。作为测试人员,我们总是有许多不同的任务要做。如果您正在学习一种新工具或尝试自动化新测试,请不要同时处理多项任务。找一大块时间集中注意力。这很难,但不断切换齿轮更难。

Allow yourself to succeed. Work at a sustainable pace. Take the time to refactor as you go or you’ll end up with a mess eventually. As testers, we always have many different tasks to do. If you’re learning a new tool or trying to automate new tests, don’t multitask. Find a big block of time and focus. This is hard, but switching gears constantly is harder.

如果业务利益相关者对您的团队“完成它”不耐烦,请与他们一起分析问题。有什么风险?生产问题的成本是多少?发布 quick hack 有什么好处?它会增加多少技术债务?由自动化测试支持的可靠设计的长期投资回报是多少?每种方法将如何影响公司的盈利能力和客户满意度?无形成本如何,比如做低质量的工作对团队士气的影响?有时业务是正确的,但我们打赌您通常会发现前期投资得到回报。

If business stakeholders are impatient for your team to “just get it done,” analyze the problem with them. What are the risks? How much will a production problem cost? What are the benefits of releasing a quick hack? How much technical debt will it add? What’s the long-term return on investment of a solid design supported with automated tests? How will each approach affect company profitability and customer satisfaction? What about the intangible costs, such as the effect that doing poor-quality work has on team morale? Sometimes the business will be right, but we’re betting that you’ll usually find that up-front investment pays off.

通过实践学习

Learn by Doing

每个人都以不同的方式学习,但是当您决定了如何使测试自动化时,请投入并开始进行。在Everyday Scripting with Ruby for Teams, Testers, and You [2007] 中,Brian Marick 建议通过编写程序来学习编程。犯错误!你遇到的问题越多,你就会学到越多。让某人与您配对将有助于加快学习速度,即使你们都不熟悉该工具或语言。

Everyone learns in different ways, but when you’ve decided how you’re going to automate a test, jump in and start doing it. In Everyday Scripting with Ruby for Teams, Testers, and You [2007], Brian Marick advises to learn to program by writing a program. Make mistakes! The more problems you have, the more you’ll learn. Getting someone to pair with you will help speed up learning, even if neither one of you is familiar with the tool or the language.

如果您没有人可以配对,请与“橡皮鸭”交谈:想象一下您正在向同事描述问题。解释的过程往往可以使问题的原因跳入视野。简单地大声朗读测试可以帮助您找到其中的弱点。

If you don’t have anyone to pair with, talk to the “rubber ducky”: Imagine you’re describing the problem to a coworker. The process of explaining can often make the cause of the problem jump into view. Simply reading a test aloud to yourself can help you find the weaknesses in it.

将敏捷编码实践应用于测试

Apply Agile Coding Practices to Tests

测试与生产代码一样有价值。事实上,没有测试支持的生产代码并不多。像对待所有代码一样对待测试。将其保存在与生产代码相同的源代码控制工具中。您应该始终能够识别与特定代码版本一起使用的测试脚本的版本。

Tests are just as valuable as production code. In fact, production code isn’t much without tests to support it. Treat your tests the same way you treat all code. Keep it in the same source code control tool as your production code. You should always be able to identify the versions of test scripts that go with a particular version of code.

结对、重构、简单设计、模块化和面向对象的设计、良好的标准、尽可能保持测试独立——所有优秀代码的品质也是优秀自动化测试的品质。敏捷开发有时会被不知情的人认为是混乱或松散的,而实际上它是高度纪律的。以最严格的纪律执行您的自动化任务,分小步进行,检查每一步是否成功。如果您正在编写自动化脚本,请先编写测试,就像任何敏捷程序员编写生产代码一样。不过,请记住简单性。除非有良好的投资回报率,否则不要编写包含大量逻辑的花哨测试脚本。这些测试需要测试并且维护成本更高。尽可能指定测试而不是编写代码,并始终使用尽可能简单的方法。

Pairing, refactoring, simple design, modular and object-oriented design, good standards, keeping tests as independent as possible—all of the qualities of good code are also qualities of good automated tests. Agile development is sometimes perceived by the uninformed to be chaotic or lax, when in fact it is highly disciplined. Undertake your automation tasks with the greatest discipline, proceeding in small steps, checking in each step that succeeds. If you’re programming automated scripts, write them test-first, just as any agile programmer would write production code. Keep simplicity in mind, though. Don’t write fancy test scripts with lots of logic unless there’s a good ROI. Those tests need testing and cost more to maintain. Specify tests when you can instead of coding them, and always go with the simplest approach possible.

我们怎么强调都不为过:测试自动化是一项团队工作。不同团队成员的不同经验、技能和观点可以一起工作,以提出最佳的自动化方法。创新——有创意。做适合你独特情况的事情,不管“常识”怎么说。

We can’t emphasize it enough: Test automation is a team effort. The varying experience, skills, and perspectives of different team members can work together to come up with the best approach to automation. Innovate—be creative. Do what works for your unique situation, no matter what the “common wisdom” says.

自动化工具只是其中的一块拼图。测试环境和测试数据是必不可少的组成部分。接下来我们来看测试数据。

Automation tools are just one piece of the puzzle. Test environments and test data are essential components. Let’s look at test data next.

为测试提供数据

Supplying Data for Tests

无论我们使用什么工具来自动化测试,测试都需要数据来处理。理想情况下,他们需要与生产数据匹配的真实数据。然而,生产数据库通常包含大量数据,而且它们可能非常复杂。此外,数据库访问会以指数方式减慢测试速度。就像很多敏捷测试一样,它是一种平衡行为。

No matter what tool we use to automate tests, the tests need data to process. Ideally, they need realistic data that matches production data. However, production databases usually contain lots and lots of data, and they can be highly complex. Also, database access slows down tests exponentially. Like so much of agile testing, it’s a balancing act.

数据生成工具

Data Generation Tools

在我们撰写本书时,有几种很酷的工具可用于为各种输入字段和边界条件生成测试数据。Data Generator、databene benerator、testgen、Datatect 和 Turbo Data 等开源和商业工具可用于生成平面文件或直接向数据库表生成数据。这些工具可以生成种类繁多的不同类型的数据,例如姓名和地址。

As we write this book, there are several cool tools available to generate test data for all kinds of input fields and boundary conditions. Open source and commercial tools such as Data Generator, databene benerator, testgen, Datatect, and Turbo Data are available to generate flat files or generate data directly to database tables. These tools can generate huge varieties of different types of data, such as names and addresses.

使用 Ruby 或 Python 等脚本语言、Fit 或 FitNesse 等工具或 shell 脚本,使用自行开发的脚本生成测试数据也相当容易。

It’s also fairly easy to generate test data with a home-grown script, using a scripting language such as Ruby or Python, a tool such a Fit or FitNesse, or a shell script.

丽莎的故事

Lisa’s Story

我们的 Watir 脚本创建随机测试数据输入,以确保它们可重新运行(它们不太可能两次创建具有相同 SSN 的员工),并提供各种数据和场景。创建新退休计划的脚本会生成包含大约 200 种不同选项组合的计划。测试贷款的脚本随机生成贷款的频率、原因和期限,并验证预期的付款是否正确。

Our Watir scripts create randomized test data inputs, both to ensure they are rerunnable (they’re unlikely to create an employee with the same SSN twice), and to provide a variety of data and scenarios. The script that creates new retirement plans produces plans with about 200 different combinations of options. The script that tests taking out a loan randomly generates the frequency, reason, and term of the loan, and verifies that the expected payment is correct.

我们有实用程序脚本来创建用于测试上传的逗号分隔文件。例如,系统中有几个地方可以上传包含新员工信息的人口普查文件。如果我需要一个包含 1,000 名新员工的测试文件,并且对退休计划进行随机投资分配,我可以简单地运行脚本并指定员工数量、他们投资的共同基金和文件名。每条记录都会有一个随机生成的社会安全号码、姓名、地址、受益人、工资递延金额和投资基金分配。下面是生成投资计算的代码片段。

We have utility scripts to create comma-separated files for testing uploads. For example, there are several places in the system that upload census files with new employee information. If I need a test file with 1,000 new employees with random investment allocations to a retirement plan, I can simply run the script and specify the number of employees, the mutual funds they’re investing in, and the file name. Each record will have a randomly generated Social Security Number, name, address, beneficiaries, salary deferral amounts, and investment fund allocations. Here’s a snippet of the code to generate the investment calculations.

# 33% 的时间最大化选择的基金数量,33% 的时间

选择一个基金,33% 的时间从 2-4 个基金中选择

    fund_hash =  case  rand( 3 )

      when 0 : a.get_random_allocations( @fund_list .clone)当 1 时:a.get_random_allocations(@fund_list .clone,  1当 2 时:a.min_percent  = 8;a.get_random_allocations(@fund_list .clone , rand( 3 ) +  2 ) end     emp['fund_allocations'] = fund_hash_to_string(fund_hash)

      





    

# 33% of the time maximize the number of funds chosen, 33% of the time

# select a single fund, and 33% of the time select from 2-4 funds

    fund_hash = case rand(3)

      when 0: a.get_random_allocations(@fund_list.clone)

      when 1: a.get_random_allocations(@fund_list.clone, 1)

when 2: a.min_percent = 8;

a.get_random_allocations(@fund_list.clone, rand(3) + 2)

    end

    emp['fund_allocations'] = fund_hash_to_string(fund_hash)

像这样的脚本有双重用途,既可以作为涵盖许多不同场景的回归测试,也可以作为创建测试数据和构建测试场景的探索性测试工具。学习写作并不难(参见本章前面的“边做边学”部分)。

Scripts like these have dual uses, both as regression tests that cover a lot of different scenarios and exploratory test tools that create test data and build test scenarios. They aren’t hard to learn to write (see the section “Learning by Doing” earlier in this chapter).

——丽莎

—Lisa

生成测试数据的脚本和工具不必很复杂。例如,PerlClip 只是将文本生成到 Windows 剪贴板中,以便可以将其粘贴到需要的地方。任何能够消除足够单调乏味、让您发现有关应用程序的潜在问题的解决方案都值得尝试。“可能可行的最简单的事情”绝对适用于为测试创建数据。您希望让您的测试尽可能简单和快速。

Scripts and tools to generate test data don’t have to be complex. For example, PerlClip simply generates text into the Windows clipboard so it can be pasted in where needed. Any solution that removes enough tedium to let you discover potential issues about the application is worth trying. “The simplest thing that could possibly work” definitely applies to creating data for tests. You want to keep your tests as simple and fast as possible.

避免数据库访问

Avoid Database Access

您的首选测试应该尝试拥有可以完全在内存中运行的测试。他们仍然需要设置和拆除测试数据,但数据不会存储在数据库中。每个测试都是独立的,并且运行速度与任何测试一样快。数据库访问意味着 I/O 和磁盘本来就很慢。每次读取数据库都会减慢您的测试运行速度。如果您的目标是向团队提供快速反馈,那么您希望您的测试尽快运行。诸如内存数据库之类的假对象可以让测试做它需要做的事情,并且仍然提供即时反馈。

Your first choice for testing should try to have tests that can run completely in-memory. They will still need to set up and tear down test data, but the data won’t store in a database. Each test is independent and runs as quickly as any test could. Database access means I/O and disks are inherently slow. Every read to the database slows down your test run. If your goal is to give fast feedback to the team, then you want your tests to run as quickly as possible. A fake object such as an in-memory database lets the test do what it needs to do and still give instant feedback.

丽莎的故事

Lisa’s Story

我们的构建流程之一仅运行单元级测试,我们尽量将其运行时间控制在 8 分钟以内,以获得最佳反馈。在大多数情况下,测试用假对象代替真实数据库。实际上测试数据库层的测试,例如将数据持久化到数据库,使用一个小模式,其中包含最初从生产数据库复制的规范数据。数据是真实的,但数量少,访问速度更快。

One of our build processes runs only unit-level tests, and we try to keep its runtime less than eight minutes, for optimum feedback. The tests substitute fake objects for the real database in most cases. Tests that are actually testing the database layer, such as persisting data to the database, use a small schema with canonical data originally copied from the production database. The data is realistic, but the small amount makes access faster.

在功能测试级别,我们的 FitNesse 测试装置尽可能在内存中构建数据。这些测试运行很快,结果几乎是瞬间出现。当我们需要测试数据库层时,或者如果我们需要测试无法独立于数据库层访问的遗留代码时,我们通常会编写 FitNesse 测试,这些测试使用自行开发的数据夹具设置和拆除自己的数据。这些测试是必要的,但它们运行缓慢且维护成本高昂,因此我们将它们保持在给我们信心所需的绝对最低限度。我们希望运行面向业务的测试的构建能够在几个小时内提供反馈,以保持我们的工作效率。

At the functional test level, our FitNesse test fixtures build data in-memory wherever possible. These tests run quickly, and the results appear almost instantaneously. When we need to test the database layer, or if we need to test legacy code that’s not accessible independently of the database layer, we usually write FitNesse tests that set up and tear down their own data using a home-grown data fixture. These tests are necessary, but they run slowly and are expensive to maintain, so we keep them to the absolute minimum needed to give us confidence. We want our build that runs business-facing tests to provide feedback within a couple of hours in order to keep us productive.

——丽莎

—Lisa

可以在xUnit 测试模式中找到各种类型的测试替身的综合解释和示例。请参阅参考书目以获取更多信息以及用于处理测试存根以及模拟和伪造对象的工具。

Comprehensive explanations and examples of various types of test doubles can be found in xUnit Test Patterns. See the bibliography for more information on that and tools for working with test stubs and with mock and fake objects.

因为很难获得测试自动化的牵引力,所以很容易说“好吧,我们有一些测试,它们确实需要几个小时才能运行,但总比没有测试好。” 数据库访问是导致测试缓慢的主要原因。继续采取小步骤尽可能地伪造数据库,并在不涉及数据库的情况下测试尽可能多的逻辑。如果这很困难,请重新评估您的系统架构,看看它是否可以更好地组织以进行测试。

Because it’s so difficult to get traction on test automation, it would be easy to say “OK, we’ve got some tests, and they do take hours to run, but it’s better than no tests.” Database access is a major contributor to slow tests. Keep taking small steps to fake the database where you can, and test as much logic as possible without involving the database. If this is difficult, reevaluate your system architecture and see if it can be organized better for testing.

DbFit、NdbUnit等工具可以简化数据库测试,实现测试驱动的数据库开发;请参阅参考书目以获取更多资源。

Tools such as DbFit and NdbUnit can simplify database testing and enable test-driven database development; see the bibliography for more resources.

如果您正在测试代码中的业务逻辑、算法或计算,您会对代码本身在给定特定输入时的行为感兴趣;你不关心数据从哪里来,只要它能准确地代表真实数据。如果是这种情况,请构建作为测试的一部分并且可以在内存中访问的测试数据,并让生产代码从中运行。模拟数据库访问和对象,并专注于测试的目的。测试不仅运行得更快,而且更容易编写和维护。

If you’re testing business logic, algorithms, or calculations in code, you’re interested in the behavior of the code itself given certain inputs; you don’t care where the data comes from as long as it accurately represents real data. If this is the case, build test data that is part of the test and can be accessed in memory, and let the production code operate from that. Simulate database access and objects, and focus on the purpose of the test. Not only will the tests run faster, but they’ll be easier to write and maintain.

为测试生成数据时,请尽可能使用反映测试意图的值。除非您完全确信每个测试都是独立的,否则请为每个测试生成唯一的测试值。例如,使用时间戳作为字段值的一部分。独特的数据是另一个安全网,可以防止测试与杂散数据相互感染。当您需要大量数据时,请尝试随机生成数据,但始终在测试结束时对其进行清理,以免其渗入下一个测试。我们认识到,有时您需要测试非常特定类型的数据。在这些情况下,随机生成的数据会破坏测试的目的。但是您可以使用足够的随机化来确保每个测试都有唯一的输入。

When generating data for a test, use values that reflect the intent of the test, where possible. Unless you’re completely confident that each test is independent, generate unique test values for each test. For example, use timestamps as part of the field values. Unique data is another safety net to keep tests from infecting each other with stray data. When you need large amounts of data, try generating the data randomly, but always clean it up at the end of the test so that it doesn’t bleed into the next test. We recognize that sometimes you need to test very specific types of data. In these cases, randomly generated data would defeat the purpose of the test. But you may be able to use enough randomization to ensure that each test has unique inputs.

当数据库访问不可避免甚至需要时

When Database Access Is Unavoidable or Even Desirable

如果被测系统对数据库的依赖很大,这个自然要测试。如果您正在测试的代码从数据库读取和/或写入数据库,那么在某些时候您需要对其进行测试,并且您可能至少需要一些回归测试来验证代码的数据库层。

If the system under test relies heavily on the database, this naturally has to be tested. If the code you’re testing reads from and/or writes to the database, at some point you need to test that, and you’ll probably want at least some regression tests that verify the database layer of code.

每个测试的设置/拆卸数据

我们首选的方法是让每个测试都将它需要的数据添加到测试模式中,对数据进行操作,在数据库中验证结果,然后删除所有这些测试数据,以便可以重新运行测试而不影响其他后续测试。这支持了测试相互独立的想法。

Our preferred approach is to have every test add the data it needs to a test schema, operate on the data, verify the results in the database, and then delete all of that test data so the test can be rerun without impacting other subsequent tests. This supports the idea that tests are independent of each other.

丽莎的故事

Lisa’s Story

我们使用通用数据夹具,让编写测试的人员指定数据库表、列和列的值,以便添加数据。另一个通用数据查找装置让我们输入表名和 SQL where 子句来验证实际的持久数据。我们还可以使用通用数据夹具通过表名和键值删除数据。图 14-3显示了一个使用数据夹具在数据库中构建测试数据的表的示例。它填充表具有指定列和值的“所有基金”。对于我们这些编写测试用例的人来说,用我们需要的所有数据填充表很容易。

We use a generic data fixture that lets the person writing the test specify the database table, columns, and values for the columns in order to add data. Another generic data lookup fixture lets us enter a table name and SQL where clause to verify the actual persisted data. We can also use the generic data fixture to delete data using the table name and a key value. Figure 14-3 shows an example of a table that uses a data fixture to build test data in the database. It populates the table “all fund” with the specified columns and values. It’s easy for those of us writing test cases to populate the tables with all of the data we need.

图 14-3使用数据夹具在数据库中构建测试数据的表示例

Figure 14-3 Example of a table using a data fixture to build test data in the database

图像

请注意,我们用于这些测试的模式已删除了大部分约束,因此我们只需填充与被测试功能相关的表和列。这也使维护更容易一些。缺点是测试不太现实,但使用其他工具的测试会在现实环境中验证功能。

Note that the schemas we use for these tests have most of their constraints removed, so we only have to populate the tables and columns pertinent to the functionality being tested. This makes maintenance a little easier, too. The downside is that the test is a bit less realistic, but tests using other tools verify the functionality with a realistic environment.

以这种方式创建测试数据的缺点是,每当在数据库中进行更改时,例如具有所需值的新列,填充该表的测试中的所有数据夹具表都必须更改。这些测试编写和维护起来可能很麻烦,因此我们只在绝对需要时才使用它们。我们尝试设计测试以降低维护成本。例如,图 14-3中的数据夹具位于“包含”库中,可以包含到需要它的测试中。假设我们添加了一个新列“fund_category”。我们只需要将它添加到这个“include”表中,而不是在使用它的 20 个不同测试中。

The downside to creating test data this way is that whenever a change is made in the database, such as a new column with a required value, all of the data fixture tables in the tests that populate that table will have to be changed. These tests can be burdensome to write and maintain, so we only use them when absolutely needed. We try to design the tests to keep maintenance costs down. For example, the data fixture in Figure 14-3 is in an “include” library and can be included into the tests that need it. Let’s say we add a new column, “fund_category.” We only need to add it to this “include” table, rather than in 20 different tests that use it.

——丽莎

—Lisa

规范数据

另一种选择是拥有可以使用规范或种子数据库中的数据快速刷新的测试模式。这个想法是这个种子数据是真实生产数据的代表性样本。因为它是少量数据,所以每次需要运行一套回归测试时都可以快速重建它。

Another alternative is having test schemas that can quickly be refreshed with data from a canonical or seed database. The idea is that this seed data is a representative sample of real production data. Because it’s a small amount of data, it can be quickly rebuilt each time a suite of regression tests needs to be run.

这种方法还增加了运行测试所需的时间,但在回归套件开始时只需几分钟,而不是从每个单独的测试中抽出时间。这些测试仍然比不访问数据库的测试慢,但比必须费力地填充每个表中的每一列的测试快。

This approach also increases the time it takes to run tests, but it’s just a few minutes at the start of the regression suite rather than taking time out of each individual test. The tests will still be slower than tests that don’t access the database, but they’ll be faster than tests that have to laboriously populate every column in every table.

规范数据有很多用途。测试人员和程序员可以有自己的测试模式,可以随意刷新。他们可以进行手动和自动测试,而无需踩到其他任何人的测试。如果仔细选择数据,数据将比每个测试可以为自己构建的有限数据量更真实。

Canonical data has many uses. Testers and programmers can have their own test schema to refresh at will. They can conduct both manual and automated tests without stepping on anyone else’s testing. If the data is carefully chosen, the data will be more realistic than the limited amount of data each test can build for itself.

当然,与几乎所有事物一样,它也有缺点。规范数据可能很难跟上。当您需要新的测试场景时,您必须确定可用的生产数据,或者构建您需要的数据并将其添加到种子模式中。出于安全原因,您必须清理数据,掩盖真实人员的识别特征,使其无害。每次添加将表或列添加到生产数据库,您必须相应地更新您的测试模式。您可能必须每年前滚日期敏感数据,或进行其他大规模维护。您必须仔细选择哪些表应该刷新,哪些表不需要刷新,例如查找表。如果您必须添加数据以增加测试覆盖率,则刷新将花费更长的时间,从而增加触发它的构建过程的时间。正如我们一直强调的那样,您的自动化构建及时提供反馈非常重要,因此越来越长的数据库刷新会延长您的反馈周期。您还会失去与规范数据的测试独立性,因此如果一个测试失败,其他测试可能也会效仿。

Of course, as with practically everything, there’s a downside. Canonical data can be a pain to keep up. When you need new test scenarios, you have to identify production data that will work, or make up the data you need and add it to the seed schema. You have to scrub the data, mask real peoples’ identifying characteristics, making it innocuous for security reasons. Every time you add a table or column to the production database, you must update your test schemas accordingly. You might have to roll date-sensitive data forward every year, or do other large-scale maintenance. You have to carefully select which tables should be refreshed and which tables don’t need refreshing, such as lookup tables. If you have to add data to increase test coverage, the refresh will take longer to do, increasing the time of the build process that triggers it. As we’ve been emphasizing, it’s important that your automated builds provide feedback in a timely manner, so longer and longer database refreshes lengthen your feedback cycle. You also lose the test independence with canonical data, so if one test fails, others may follow suit.

Lisa 的团队成员运行他们的 GUI 测试套件,并且他们针对模式的一些功能回归测试每次运行都使用规范数据刷新。在极少数情况下,由于对种子数据的错误更新,测试会意外失败。决定是否向前“滚动”数据,例如,将 2008 年的行变成 2009 年的行,这会让人头疼。到目前为止,使用规范数据的投资回报率已经为团队所接受。Janet 目前的团队还使用种子数据对本地构建进行“中间层”测试。它适用于开发周期中的快速反馈。但是,测试环境和暂存环境使用生产数据的迁移副本。缺点是回归测试只能在构建的本地副本上运行。风险很低,因为他们实行“一次构建,多次部署”。

Lisa’s team members run their GUI test suites and some of their functional regression tests against schemas refreshed each run with canonical data. On rare occasions, tests fail unexpectedly because of an erroneous update to the seed data. Deciding whether to “roll” data forward, so that, for example, 2008’s rows become 2009’s rows, gets to be a headache. So far, the ROI on using canonical data has been acceptable for the team. Janet’s current team also uses seed data for its “middle layer” testing on local builds. It works well for fast feedback during the development cycle. However, the test environment and the staging environments use a migrated copy of production data. The downside is that the regression tests can only be run on local copies of the build. The risk is low because they practice “build once, deploy to many.”

类生产数据

测试尽可能像生产系统的能力对于大多数软件开发团队来说是必不可少的。但是,针对生产数据库的副本运行一套自动回归测试可能会运行得太慢而无法提供有用的反馈。此外,当您带来新副本以保持最新时,您不能真正依赖任何保持稳定的数据。通常,当您谈论功能或端到端测试时,生产数据库的克隆对于手动探索性测试最为有用。

The ability to test a system that is as much like production as possible is essential to most software development teams. However, running a suite of automated regression tests against a copy of a production database would probably run too slowly to be useful feedback. Besides, you couldn’t really depend on any data remaining stable as you bring over new copies to stay up-to-date. Generally, when you’re talking about functional or end-to-end testing, a clone of the production database is most useful for manual exploratory testing.

压力、性能和负载测试是自动化密集型的,需要一个能够密切模拟生产的环境,以便提供可以转化为实际操作的结果。可用性、安全性和可靠性是需要类似生产系统的测试的其他示例,尽管它们可能不涉及太多自动化。

Stress, performance, and load testing, which are automation–intensive, need an environment that closely simulates production in order to provide results that can translate to actual operations. Usability, security, and reliability are other examples of testing that needs a production-like system, although they may not involve much automation.

总有一个权衡;您的生产数据库可能很大,因此它既昂贵又缓慢,但它提供了最准确的可用测试数据。如果您的组织能够负担得起硬件和软件来存储用于测试目的的生产数据的多个副本,那么这是理想的选择。小公司可以具有资源限制,可能会限制可以存储在测试和暂存环境中的数据量。在这种情况下,您需要决定可以支持多少测试数据,并计划如何复制足够多的相关数据以使测试能够代表“现实生活”中使用的数据。或者您可以考虑对硬件进行投资,它的成本每天都在下降,以支持真正的生产风格环境。否则,您的测试结果可能会产生误导。正如我们在规范数据中提到的,您可能需要在使用数据之前清理数据。

There is always a trade-off; your production database might be huge, so it is expensive and slow, but it provides the most accurate test data available. If your organization can afford hardware and software to store multiple copies of production data for testing purposes, this is ideal. Small companies may have resource constraints that might limit the amount of data that can be stored in test and staging environments. In this case, you’ll need to decide how much test data you can support and plan how to copy enough relevant data to make the test representative of what’s used in “real life.” Or you may consider making the investment in hardware, which is getting less expensive every day, to support a real production style environment. Otherwise, your test results might be misleading. As we mentioned with the canonical data, you may need to scrub the data before using it.

数据迁移

数据迁移需要针对真实数据库进行测试。数据库升级脚本需要针对真实数据和数据库模式的最新已知版本运行。

Data migration needs to be tested against a real database. The database upgrade scripts need to be run against real data and against the last known release of the database schema.

了解您的需求

Understand Your Needs

如果您了解测试的目的,则可以更好地评估您的需求。例如,如果您不需要直接测试存储过程或 SQL 查询的速度,请考虑使用内存数据库等工具,它们的工作方式与真实数据库一样,但可以大大加快您的测试速度。当您需要模拟实际生产环境时,将整个生产数据库,如有必要。快速反馈是目标,因此请尽可能有效地平衡测试真实场景和发现缺陷。

If you understand the purpose of your tests, you can better evaluate your needs. For example, if you don’t need to test stored procedures or SQL queries directly for speed, consider tools such as in-memory databases, which work just like real databases but greatly speed up your tests. When you need to simulate the actual production environment, make a copy of the entire production database, if necessary. Quick feedback is the goal, so balance testing realistic scenarios with finding defects as efficiently as possible.

评估自动化工具

Evaluating Automation Tools

选择自动化工具的第一步是列出该工具需要为您做的所有事情。让我们考虑一下如何确定您的测试工具要求。

The first step in choosing an automation tool is to make a list of everything the tool needs to do for you. Let’s consider how you can decide on your test tool requirements.

确定自动化工具的要求

Identifying Requirements for Your Automation Tool

在决定要应对的下一个自动化挑战之后,请考虑您的工具需求。您已经拥有哪些工具?如果您需要额外的,您可能想要与您现有的测试和开发基础架构很好地集成的东西。您是否需要一种工具来轻松集成到持续构建过程中?您现有的硬件是否支持您需要执行的自动化操作?设置第二个构建过程以运行功能测试可能需要额外的机器。

After deciding on the next automation challenge to tackle, think about your tool needs. What tools do you already have? If you need additional ones, you probably want something that integrates well with your existing testing and development infrastructure. Do you need a tool to easily integrate into the continuous build process? Will your existing hardware support the automation you need to do? Setting up a second build process to run functional tests may require additional machinery.

谁将使用您希望实施的测试工具?非程序员会编写测试用例吗?您的程序员是否也想要一个他们觉得舒服的工具?您是否有需要协作的分布式团队成员?

Who’s going to use the test tool you’re hoping to implement? Will non-programmers be writing test cases? Do your programmers want a tool they feel comfortable with as well? Do you have distributed team members who need to collaborate?

谁将自动化和维护测试?您团队中已有的技能很重要。您需要多少时间来安装工具并学习如何使用它?如果您的应用程序是用 Java 编写的,那么使用 Java 编写脚本的工具可能是最合适的。团队成员是否有使用特定工具的经验?是否有一个单独的测试团队在某种工具方面具有专业知识?如果您正在开始向敏捷开发过渡,并且您已经拥有一个测试自动化团队,那么利用他们的专业知识并继续使用他们知道的工具可能是有意义的。

Who will be automating and maintaining the tests? The skills already on your team are important. How much time do you have to get a tool installed and learn how to use it? If your application is written in Java, a tool that uses Java for scripting may be the most appropriate. Do team members have experience with particular tools? Is there a separate test team with expertise in a certain tool? If you’re starting the transition to agile development and you already have a team of test automators, it may make sense to leverage their expertise and keep using the tools they know.

您的工具要求取决于您的开发环境。如果您正在测试 Web 应用程序,而您选择的工具不支持 SSL 或 AJAX,您可能会遇到问题。并非每个测试工具都可以测试 Web 服务应用程序。嵌入式系统测试可能又需要不同的工具。第 12 章测试象限总结”中的案例研究展示了一种使用 Ruby 测试嵌入式应用程序的方法。

Your tool requirements are dependent on your development environment. If you’re testing a web application, and the tool you choose doesn’t support SSL or AJAX, you may have a problem. Not every test tool can test web services applications. Embedded system testing can need different tools again. The case study in Chapter 12, “Summary of Testing Quadrants,” shows one way to use Ruby to test an embedded application.

当然,您正在自动化的测试类型是关键。安全测试可能需要高度专业化的工具。有许多现有的开源和性能的供应商工具,所以选择一个的工作并不是压倒性的。当您掌握一项挑战时,您将为下一项做好更充分的准备。Lisa 的团队花了几年时间在单元、集成和功能级别开发强大的回归测试套件。性能测试是他们的下一个痛点。从早期的自动化工作中吸取的教训帮助他们更好地确定测试工具的要求,例如报告结果的简便性、与现有框架的兼容性和脚本语言。

Of course, the type of testing you’re automating is key. Security testing probably needs highly specialized tools. There are many existing open source and vendor tools for performance, so the job of selecting one isn’t overwhelming. As you master one challenge, you’ll be better prepared for the next. It took Lisa’s team a couple of years to develop robust regression test suites at the unit, integration, and functional levels. Performance testing was their next area of pain. Lessons learned from the earlier automation efforts helped them do a better job of identifying requirements for a test tool, such as ease of reporting results, compatibility with existing frameworks, and scripting language.

编写一份包含所有工具要求的清单。其中一些可能会相互冲突或相互矛盾——“该工具需要足够简单,以便客户可以指定测试”或“测试应该易于自动化”。把它们写下来,这样你就可以找到合适的平衡点。然后开始做你的研究。

Write a checklist that captures all your tool requirements. Some of them might conflict with or contradict each other—“The tool needs to be easy enough so that customers can specify tests” or “The tests should be easy to automate.” Write them down so you can find the right balance. Then start doing your research.

一次一个工具

One Tool at a Time

您将需要不同的工具来满足不同的目的。实施新工具并学习使用它们的最佳方法可能很快就会让人不知所措。一次尝试一种工具,解决您最大的疼痛区域。给它足够的时间进行公平的审判并评估结果。如果它对您有用,请先掌握该工具,然后再继续处理下一个疼痛区域和下一个工具。多任务处理可能适用于某些情况,但新技术需要全神贯注。

You’re going to need different tools to serve different purposes. Implementing new tools and learning the best way to use them can get overwhelming pretty quickly. Try one tool at a time, addressing your greatest area of pain. Give it enough time for a fair trial and evaluate the results. If it’s working for you, master that tool before you go on to the next area of pain and the next tool. Multitasking might work for some situations, but new technology demands full attention.

当您确定了一种工具来满足特定需求时,请退后一步,看看您还需要什么。您的团队面临的下一个自动化挑战是什么?您刚刚为另一个目的选择的工具是否也能满足该需求,或者您是否需要开始新的选择过程?

When you’ve settled on a tool to address a particular need, take a step back and see what else you need. What’s the next automation challenge facing your team? Will the tool you just selected for another purpose work for that need, too, or do you need to start a new selection process?

如果您决定在自己的组织之外寻找工具,那么第一步就是找时间尝试一些工具。从一些基础研究开始:Internet 搜索、关于工具的文章和其他出版物以及邮件列表是获取想法的好地方。编制要考虑的工具列表。如果您的团队使用 wiki 或在线论坛工具,请发布有关工具的信息并开始讨论优缺点。

If you’ve decided to look outside your own organization for tools, the first step is to find time to try some out. Start with some basic research: Internet searches, articles and other publications about tools, and mailing lists are good places to get ideas. Compile a list of tools to consider. If your team uses a wiki or online forum tool, post information about tools and start a discussion about pros and cons.

参考书目包含有助于工具搜索和评估的网站。

The bibliography contains websites that help with tool searches and evaluation.

评估工具的预算时间。一些团队每隔几个月就会进行一次“工程冲刺”或“重构迭代”,而不是交付业务优先考虑的故事,他们开始致力于减少技术债务、升级工具版本和试用新工具。如果您的团队还没有这些,请向您的管理层提出申请以获取它们。减少你的技术债务和建立良好的测试基础设施将提高您未来的速度和探索性测试的空闲时间。如果您从来没有时间让代码更易于维护或升级工具,技术债务就会拖累您的速度,直至停滞不前。

Budget time for evaluating tools. Some teams have an “engineering sprint” or “refactoring iteration” every few months where, rather than delivering stories prioritized by the business, they get to work on reducing technical debt, upgrading tool versions, and trying out new tools. If your team doesn’t have these yet, make a case to your management to get them. Reducing your technical debt and establishing a good testing infrastructure will improve your velocity in the future and free time for exploratory testing. If you never have time to make code easier to maintain or upgrade tools, technical debt will drag down your velocity until it comes to a halt.

当你有一个可能满足你要求的工具列表时,将可能性缩小到一两个,学习如何充分使用每个工具来尝试它,然后做一个尖峰:尝试一个简单但有代表性的场景,你可以扔掉. 根据要求评估结果。使用回顾来考虑利弊。

When you have a list of tools that may meet your requirements, narrow the possibilities down to one or two, learn how to use each one well enough to try it, and do a spike: Try a simple but representative scenario that you can throw away. Evaluate the results against the requirements. Use retrospectives to consider pros and cons.

您需要哪些资源来实施和使用该工具?该工具将对团队的生产力和速度产生什么影响?它会带来什么风险?从长远来看,它会让你做哪些你现在做不到的事情?

What resources do you need to implement and use the tool? What impact will the tool have on the team’s productivity and velocity? What risks does it pose? What will it allow you to do in the long term that you can’t do now?

选择你的最佳候选人并承诺尝试一段时间——足够长的时间来获得一些能力。确保尝试所有关键任务功能。例如,如果您的应用程序大量使用 Ajax,请确保您可以使用该工具进行自动化测试。在回顾中,看看哪些有效,哪些无效。对这样的想法持开放态度,即它可能不正确并且您必须将其丢弃并重新开始。不要觉得你必须继续使用这个工具,因为你已经在它上面投入了很多。

Pick your top candidate and commit to trying it for some period of time—long enough to get some competency with it. Make sure you try all your mission-critical functionality. For example, if your application uses a lot of Ajax, make sure you can automate tests using the tool. In retrospectives, look at what worked and what didn’t. Be open to the idea that it might not be right and that you have to throw it out and start over. Don’t feel you have to keep on with the tool because you have so much invested in it already.

我们都知道,没有可以解决所有自动化问题的“灵丹妙药”。降低你的期望,敞开心扉。创造性的解决方案既依赖于艺术,也依赖于科学。

We all know that there’s no “silver bullet” that can solve all your automation problems. Lower your expectations and open your mind. Creative solutions rely on art as much as science.

丽莎的故事

Lisa’s Story

在进行性能测试工具搜索时,我们求助于敏捷测试邮件列表以寻求建议。许多人提供了他们的经验,有些人甚至提出帮助学习和实施工具。我们搜索了一种使用 Java 编写脚本的工具,它具有最小的学习曲线,并以有用的图形格式显示结果。我们在团队 wiki 上列出了工具及其优缺点。我们为试运行预留了时间。Lisa 的同事 Mike Busse 试用了前两名候选人,并向团队其他成员展示了亮点。一个工具是由团队一致选择的,并被证明是一个很好的选择。

When conducting the performance test tool search, we turned to an agile testing mailing list for suggestions. Many people offered their experiences, and some even offered to help learning and implementing a tool. We searched for a tool that used Java for scripting, had a minimal learning curve, and presented results in a useful graphical format. We listed tools and their pros and cons on the team wiki. We budgeted time for trial runs. Lisa’s coworker, Mike Busse, tried the top two candidates and showed highlights to the rest of the team. A tool was chosen by team consensus and has proven to be a good fit.

——丽莎

—Lisa

第 11 章,“使用面向技术的测试来评价产品”,展示了所选性能测试工具 JMeter 产生的结果示例。

Chapter 11, “Critiquing the Product Using Technology-Facing Tests, shows an example of the results produced by the performance test tool chosen, JMeter.

选择工具

Choosing Tools

我们很幸运拥有范围广泛且不断增长的工具集可供选择:本土、开源、供应商工具,或组合any,都是可行的选择。有这么多选择,诀窍是知道在哪里寻找并找时间试用工具,看看它们是否符合您的要求。因为我们无法预测未来,所以可能很难判断每个潜在解决方案的投资回报率,但评估它们的迭代方法有助于找到正确的解决方案。

We’re lucky to have an already vast range and ever-growing set of tools to choose from: home-grown, open source, vendor tools, or a combination of any, are all viable alternatives. With so many choices, the trick is knowing where to look and finding time to try tools out to see if they fit your requirements. Because we can’t predict the future, it may be hard to judge the ROI of each potential solution, but an iterative approach to evaluating them helps get to the right one.

你应该自己种植吗?

您的应用程序是否存在独特的测试挑战,例如嵌入式软件或与外部系统的集成?团队成员是否具备编写自己的测试框架或在现有开源工具之上构建测试框架的技能、时间和意愿?如果是这样,本土测试工具可能是最合适的。

Does your application present unique testing challenges, such as embedded software or integration with outside systems? Do team members have the skills, time, and inclination to write their own test framework or build one on top of an existing open source tool? If so, home-grown test tools may be the best fit.

敏捷开发的一个令人高兴的结果(或者可能是一个主要的成功因素)是许多程序员被“测试感染”。今天的开发工具和语言使自动化框架更容易构建。Ruby、Groovy、Rails 以及许多语言和框架都适合自动化。可以利用现有的开源工具,例如 Fit 和 HtmlUnit,并在它们之上构建自定义框架。

A happy result (or perhaps a major success factor) of agile development is that many programmers are “test infected.” Today’s development tools and languages make automation frameworks easier to build. Ruby, Groovy, Rails, and many languages and frameworks lend themselves to automation. Existing open source tools such as Fit and HtmlUnit can be leveraged, with custom frameworks built on top of them.

本土工具有很多优势。他们绝对是程序员友好的。如果您的团队正在编写自己的自动化框架,它们将根据您的开发和客户团队的需求进行精确定制,并与您现有的构建流程和其他基础设施集成——您可以使它们像以前一样易于执行和解释结果你需要。

Home-grown tools have many advantages. They’re definitely programmer-friendly. If your team is writing its own automation frameworks, they’ll be precisely customized to the needs of your development and customer teams, and integrated with your existing build process and other infrastructure—and you can make them as easy to execute and interpret results as you need.

当然,本土并不意味着免费。一个小团队可能没有带宽来编写和支持工具以及开发生产代码。具有独特需求的大型组织可能能够组建一个自动化专家团队,他们可以与测试人员、客户、程序员和其他人协作。如果您的需求如此独特以至于没有现有工具支持它们,那么自主开发可能是您唯一的选择。

Home-grown doesn’t mean free, of course. A small team may not have the bandwidth to write and support tools as well as develop production code. A large organization with unique requirements may be able to put together a team of automation specialists who can collaborate with testers, customers, programmers, and others. If your needs are so unique that no existing tool supports them, home-grown may be your only option.

开源工具

许多编写自己工具的团队都慷慨地将它们提供给开源社区。因为这些工具是由受测试影响的程序员编写的,供应商工具无法满足他们的需求,所以它们通常是轻量级的,适合敏捷开发。其中许多工具都是测试先行开发的,您可以随源代码一起下载测试套件,使定制更容易、更安全。这些工具具有广泛的吸引力,具有对程序员和测试人员都有用的功能。这价格合适,但重要的是要记住购买价格只是任何工具成本的一小部分。

Many teams who wrote their own tools have generously made them available to the open source community. Because these tools were written by test-infected programmers whose needs weren’t met by vendor tools, they are usually lightweight and appropriate for agile development. Many of these tools are developed test-first, and you can download the test suite along with the source code, making customization easier and safer. These tools have a broad appeal, with features useful to both programmers and testers. The price is right, although it’s important to remember that purchase price is only a fraction of any tool’s cost.

并不是所有的开源工具都有很好的文档记录,培训可能是一个问题。然而,我们在许多会议和用户组会议上看到了关于使用这些工具的研讨会和教程。一些开源工具有优秀的用户手册,甚至有在线教程和预定课程。

Not all open source tools are well documented, and training can be an issue. However, we see seminars and tutorials on using these tools at many conferences and user group meetings. Some open source tools have excellent user manuals and even have online tutorials and scheduled classes available.

如果您正在考虑开源解决方案,请寻找活跃的开发人员和用户社区。是否有带宽很大的邮件列表?是否经常发布新功能?有没有办法报告错误,有人修复它们吗?其中一些工具比供应商工具具有更好的支持和更快的错误响应。为什么?编写它们的人也在使用它们,他们需要这些特性来测试他们自己的产品。

If you’re considering an open source solution, look for an active developer and user community. Is there a mailing list with lots of bandwidth? Are new features released often? Is there a way to report bugs, and does anyone fix them? Some of these tools have better support and faster response on bugs than vendor tools. Why? The people writing them are also using them, and they need those features to test their own products.

有关开源测试自动化工具的更多信息,请参阅第 9 章支持团队的面向业务的测试工具包”。

See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for more about open source test automation tools.

供应商工具

商业工具被认为是一种安全的选择。很难批评某人选择了一个已经存在多年的知名工具。他们可能会附带手册、支持和培训。对于缺乏技术背景的测试人员或其他用户,最初的提升可能会更快。有些非常健壮且功能丰富。您的公司可能已经拥有一个,并且拥有一个知道如何使用它的专家团队。

Commercial tools are perceived as a safe bet. It’s hard to criticize someone for selecting a well-known tool that’s been around for years. They’re likely to come with manuals, support, and training. For testers or other users who lack a technical background, the initial ramp-up might be faster. Some are quite robust and feature-rich. Your company may already own one and have a team of specialists who know how to use it.

尽管它们与时俱进,但供应商工具历来对程序员不友好。他们倾向于使用程序员不想花时间学习的专有脚本语言。他们也往往是重量级的。测试脚本可能很脆弱,很容易被应用程序的小改动破坏,而且维护起来很昂贵。这些工具大多是录制脚本以供后续播放。从维护的角度来看,记录/回放脚本是出了名的昂贵。

Although they are changing with the times, vendor tools are historically programmer-unfriendly. They tend to use proprietary scripting languages that programmers don’t want to spend time learning. They also tend to be heavyweight. The test scripts may be brittle, easily broken by minor changes to the application, and expensive to maintain. Most of these tools are recording scripts for subsequent playback. Record/playback scripts are notoriously costly from a maintenance perspective.

Elisabeth Hendrickson [ 2008 ] 指出,诸如此类的专用工具可能需要测试自动化专家。诸如此类的孤岛可能会与敏捷团队对抗。我们需要能够促进先测试而不是后测试开发的工具。测试工具不应成为变革的阻碍。

Elisabeth Hendrickson [2008] points out that specialized tools such as these may create a need for test automation specialists. Silos such as these can work against agile teams. We need tools that facilitate test-first, rather than test-last development. Test tools shouldn’t stand in the way of change.

请参阅参考书目,了解 Elisabeth Hendrickson 对此主题的完整讨论。

See the bibliography for a full discussion by Elisabeth Hendrickson on this subject.

如果您的人员已经精通供应商工具,并且可能仅由开发团队的一个子集或与开发分开的团队使用该工具,那么供应商工具可能很有意义。Lisa 的前两个 XP 团队使用了供应商工具并取得了一定程度的成功。

If you have people already expert in a vendor tool, and a use for a tool that might be used only by a subset of the development team or a team separate from development, a vendor tool could make lots of sense. Lisa’s first two XP teams used a vendor tool with some degree of success.

在撰写本文时,更好的功能测试工具和 IDE 不断涌现。这些通过全局搜索/替换等功能促进了测试维护任务。Twist 是作为 Eclipse IDE 插件集合实现的工具示例,因此它可以利用强大的编辑和重构功能。

As of this writing, better functional test tools and IDEs are emerging. These facilitate test maintenance tasks with features such as global search/replace. Twist is an example of a tool implemented as a collection of plug-ins to the Eclipse IDE, so it can take advantage of powerful editing and refactoring features.

敏捷友好的工具

Agile-Friendly Tools

Elisabeth Hendrickson [ 2008 ] 列出了有效的敏捷测试自动化工具的一些特征。这些工具应该:

Elisabeth Hendrickson [2008] lists some characteristics of effective agile test automation tools. These tools should:

图像支持立即开始测试自动化工作,使用测试先行的方法

Support starting the test automation effort immediately, using a test-first approach

图像将测试的本质与实现细节分开

Separate the essence of the test from the implementation details

图像支持和鼓励测试自动化代码部分的良好编程实践

Support and encourage good programming practices for the code portion of the test automation

图像支持使用真实语言和真实 IDE 编写测试自动化代码

Support writing test automation code using real languages, with real IDEs

图像促进协作

Foster collaboration

第 III 部分,“使用敏捷测试象限”,特别是第 9 章,“支持团队的面向业务的测试工具包”,包含在敏捷项目中运行良好的测试自动化工具的示例。

Part III, “Using Agile Testing Quadrants,” and particularly Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” contain examples of test automation tools that work well on agile projects.

实施自动化

Implementing Automation

在评估工具时,请考虑必须以多快的速度解决您的首要自动化需求。您将从哪里获得支持以帮助实施它?团队需要什么培训,可以投入多少时间?您必须以多快的速度升级此工具?

While you’re evaluating tools, think about how quickly your top priority automation need must be addressed. Where will you get the support to help implement it? What training does the team need, and how much time will be available to devote to it? How quickly do you have to ramp up on this tool?

在查看工具时,请牢记所有这些限制。为了在短期内实现重要的自动化,您可能不得不接受一个不如您真正想要的强大的工具。请记住,没有什么是永恒的。您可以逐步构建自动化工作。在找到工具、技能和基础设施的正确组合之前,许多团队都经历过不成功的尝试。

Keep all of these constraints in mind when you’re looking at tools. You might have to settle for a less robust tool than you really want in order to get vital automation going in the short term. Remember that nothing’s permanent. You can build your automation effort step-by-step. Many teams experience unsuccessful attempts before finding the right combination of tools, skills, and infrastructure.

良好的面向对象设计并不是构建一套可维护的自动化测试的唯一关键。您还需要经常运行测试以获得您的团队需要的反馈。无论我们选择什么工具,都必须与我们的构建过程集成。易于解释的结果应该会自动出现。

Good object-oriented design isn’t the only key to building a suite of maintainable automated tests that pay off. You also need to run the tests often enough to get the feedback your team needs.. Whatever tools we choose must be integrated with our build process. Easy-to-interpret results should come to us automatically.

我们选择的工具必须在我们的平台上工作,并且必须与我们的其他工具共享和良好地发挥作用。我们必须不断调整它们以帮助解决当前的问题。构建每天都在中断吗?也许我们需要将我们的结果与实际的交通信号灯挂钩,以建立团队对其状态的认识。面向业务的测试是否失败?应该清楚地说明什么地方失败了。我们没有额外的时间来隔离问题。

The tools we choose have to work on our platforms, and must share and play well with our other tools. We have to continually tweak them to help with our current issues. Is the build breaking every day? Maybe we need to hook our results up to an actual traffic light to build team awareness of its status. Did a business-facing test fail? It should be plain exactly what failed, and where. We don’t have extra time to spend isolating problems.

这些担忧是图片的重要组成部分,但仍然只是图片的一部分。我们需要工具来帮助我们设计模拟生产的测试环境。我们需要一些方法来保持这些测试环境独立,不受程序员可能所做的更改的影响。

These concerns are an essential part of the picture, but still only part of the picture. We need tools that help us devise test environments that mimic production. We need ways to keep these test environments independent, unaffected by changes programmers might be making.

构建测试基础设施可能是一项巨大的投资,但这是我们的敏捷团队需要进行的一项工作,以便在测试自动化方面取得飞跃。需要确定和实施硬件、软件和工具。根据贵公司的资源,这可能是一个长期项目。集思广益应对短期的方法,同时计划如何将您真正需要的基础架构组合在一起,以最大限度地降低风险、提高速度并交付尽可能最好的产品。

Building test infrastructure can be a big investment, but it’s one our agile team needs to make to get a jump on test automation. Hardware, software, and tools need to be identified and implemented. Depending on your company’s resources, this might be a long-term project. Brainstorm ways to cope in the short term, while you plan how to put together the infrastructure you really need to minimize risk, maximize velocity, and deliver the best possible product.

管理自动化测试

Managing Automated Tests

假设我们需要一种方法来找到验证特定场景的测试,了解每个测试的作用,并了解它验证的应用程序的哪一部分。也许我们需要满足从每个需求到其代码和测试的可追溯性审计要求。自动化测试需要以与生产源代码相同的方式进行维护和控制。当您标记要发布的生产代码时,验证该功能的测试需要成为标记的一部分。

Let’s say we need a way to find the test that verifies a particular scenario, to understand what each test does, and to know what part of the application it verifies. Perhaps we need to satisfy an audit requirement for traceability from each requirement to its code and tests. Automated tests need to be maintained and controlled in the same way as production source code. When you tag your production code for release, the tests that verified that functionality need to be part of the tag.

这是一个派上用场的例子。我们刚刚在开发中的代码中发现了一个问题。它是一个新问题,还是它已经潜伏在代码中一段时间​​并且以某种方式被测试遗漏了?我们可以部署生产中的标签,尝试重现问题,并调查测试未发现问题的原因。Lisa 的团队最近遇到这样一种情况,即回归套件遗漏了一个错误,因为测试模式中缺少数据库约束。如果您不将测试代码版本与生产代码版本绑定,则很难查明此类问题。

Here’s an example where that comes in handy. We just found a problem in the code under development. Is it a new problem, or has it been lurking in the code for a while and somehow missed by the test? We can deploy the tag that’s in production, try to reproduce the problem, and investigate why the tests didn’t catch it. Lisa’s team recently had a situation where the regression suite missed a bug because a database constraint was missing in the test schema. That kind of problem is hard to pinpoint if you aren’t tying your test code versions to your production code versions.

组织测试

Organizing Tests

许多工具都有自己的组织方式。例如,FitNesse 自带 wiki,具有层次结构,内置版本控制。在撰写本文时,FitNesse 开始提供对 Subversion 等源代码控制工具的支持。用其他测试工具(例如 Watir 和 Canoo WebTest)编写的脚本可以而且应该在与生产代码相同的源代码控制系统中维护,就像单元测试一样。

Many tools come with their own means of organization. For example, FitNesse comes with its own wiki, with a hierarchical organization, and built-in version control. As of this writing, FitNesse is starting to provide support for source code control tools such as Subversion. Scripts written in other test tools, such as Watir and Canoo WebTest, can and should be maintained within the same source code control system as production code, just as the unit tests are.

测试管理可帮助您的团队回答以下问题:

Test management helps your team answer questions such as the following:

图像哪些测试用例已经自动化?

Which test cases have been automated?

图像哪些还需要自动化?

Which still need automating?

图像哪些测试当前作为回归套件的一部分运行?

Which tests are currently running as part of a regression suite?

图像哪些测试涵盖哪些功能领域?

Which tests cover what functional areas?

图像XYZ 功能是如何设计的?

How is feature XYZ designed to work?

图像这个测试用例是谁写的?什么时候?最后是谁改的?

Who wrote this test case? When? Who changed it last?

图像这个测试作为回归套件的一部分多久了?

How long has this test been part of the regression suite?

因为我们编写测试的主要原因之一是指导开发,所以我们需要组织测试,以便团队中的每个人都能为每个故事找到合适的测试,并轻松确定测试涵盖的功能。因为我们使用测试作为文档,所以当存在关于系统应该如何运行的问题时,开发或客户团队中的任何人都可以快速找到特定测试是至关重要的。我们可能需要多种工具来满足不同的测试管理目标。

Because one of the primary reasons we write tests is to guide development, we need to organize tests so that everyone on the team can find the appropriate tests for each story and easily identify what functionality the tests cover. Because we use tests as documentation, it’s critical that anyone on either the development or customer team can find a particular test quickly when there’s a question about how the system should behave. We might need multiple tools to satisfy different test management goals.

很容易失去对测试脚本的控制。当测试失败时,您需要快速查明问题所在。您可能需要知道最近对测试脚本进行了哪些更改,这很容易通过源代码控制系统中可用的历史记录进行。您的客户团队还需要一种方法来跟踪项目进度,了解测试覆盖了多少代码,并可能自行运行测试。测试管理系统,就像测试本身一样,应该促进团队成员之间和不同团队之间的沟通和协作。

It’s easy to lose control of test scripts. When a test fails, you need to pinpoint the problem quickly. You may need to know what changes have been made recently to the test script, which is easy with the history available in a source code control system. Your customer team also needs a way to keep track of project progress, to understand how much of the code is covered with tests, and possibly to run tests themselves. Test management systems, like the tests themselves, should promote communication and collaboration among team members and between different teams.

组织测试结果

Organizing Test Results

参与交付软件的每个人都需要轻松访问测试和测试结果。管理测试的另一个方面是跟踪哪些测试来自先前的迭代并需要继续通过,而哪些测试正在推动当前迭代中的开发并且可能尚未通过。持续集成和构建过程运行测试以快速反馈进度并捕获回归失败。图 14-4显示了一个一目了然的测试结果报告示例。一次测试失败,并明确说明失败原因。

Everyone involved with delivering software needs easy access to tests and test results. Another aspect of managing tests is keeping track of what tests are from prior iterations and need to keep passing, versus tests that are driving development in the current iteration and may not be passing yet. A continuous integration and build process runs tests for quick feedback on progress and to catch regression failures. Figure 14-4 shows an example of a test result report that’s understandable at a glance. One test failed, and the cause of the failure is clearly stated.

图 14-4自主开发的测试管理工具的测试结果

Figure 14-4 Test results from a home-grown test management tool

图像

如果您正在通过测试推动开发,并且其中一些测试尚未通过,那么构建应该不会失败。一些团队,例如 Lisa 的团队,只是将新测试排除在集成和构建过程之外,直到它们第一次通过。在那之后,他们总是需要通过。其他团队在构建过程本身中使用规则来忽略为覆盖当前正在开发的代码而编写的测试中的失败。

If you’re driving development with tests, and some of those tests aren’t passing yet, this shouldn’t fail a build. Some teams, such as Lisa’s, simply keep new tests out of the integration and build process until they pass for the first time. After that, they always need to pass. Other teams use rules in the build process itself to ignore failures from tests written to cover the code currently being developed.

与任何测试自动化工具一样,您可以使用自己开发的、开源的或商业的系统来解决您的测试管理问题。我们在评估测试工具一节中描述的相同标准可以应用于选择测试管理方法。

As with any test automation tool, you can solve your test management problems with home-grown, open source, or commercial systems. The same criteria we described in the section on evaluating test tools can be applied to selecting a test management approach.

测试管理是应用敏捷价值观和原则以及整个团队方法的另一个领域。从简单开始。分小步进行试验,直到找到源代码控制、存储库和构建管理的正确组合,使测试和生产代码保持同步。经常评估您的测试管理方法,并确保它适应所有不同的测试用户。确定有效的方法和缺失的方法,并计划任务甚至故事以尝试使用其他工具或流程来填补任何空白。请记住保持测试管理的轻量级和可维护性,以便每个人都会使用它。

Test management is yet another area where agile values and principles, together with the whole-team approach, applies. Start simple. Experiment in small steps until you find the right combination of source code control, repositories, and build management that keeps tests and production code in synch. Evaluate your test management approach often, and make sure it accommodates all of the different users of tests. Identify what’s working and what’s missing, and plan tasks or even stories to try another tool or process to fill any gaps. Remember to keep test management lightweight and maintainable so that everyone will use it.

开始吧

Go Get Started

不要害怕得到一些东西——任何东西——到位,即使它有些不足。成功最重要的因素是刚刚开始。许多(如果不是大多数的话)成功的团队都是从一个糟糕的流程开始的,但成功地将一个不充分的流程变成了对团队成功真正重要的东西,一次一个。与敏捷测试的许多方面一样,以微小的增量进行改进是成功的关键。

Don’t be afraid to get something—anything—in place, even if it’s somewhat deficient. The most important factor in success is to just get started. Many, if not most, successful teams have started with a poor process but managed to turn an inadequate process into something truly essential to the team’s success, one piece at a time. As with so many aspects of agile testing, improving in tiny increments is the key to success.

如果你不从某个地方开始,你将永远不会获得自动化的牵引力。召集整个团队并开始实验。没有适当级别的测试自动化,您的团队就无法做到最好。您需要正确的测试自动化来频繁地交付业务价值。一两年后,您会想知道为什么您认为测试自动化如此困难。

If you don’t start somewhere, you’ll never get traction on automation. Get the whole team together and start an experiment. Without the right level of test automation, your team can’t do its best work. You need the right test automation to deliver business value frequently. A year or two from now, you’ll wonder why you thought test automation was so hard.

概括

Summary

在本章中,我们考虑了如何应用敏捷价值观、原则和实践来制定自动化战略。我们讨论了以下与自动化相关的主题:

In this chapter, we considered how to apply agile values, principles, and practices to develop an automation strategy. We discussed the following subjects related to automation:

图像使用敏捷测试象限来帮助确定您需要测试自动化的位置和时间。

Use the agile testing quadrants to help identify where you need test automation, and when you’ll need it.

图像测试自动化金字塔可以帮助您的团队在测试自动化方面做出正确的投资,从而获得最大回报。

The test automation pyramid can help your team make the right investments in test automation that will pay off the most.

图像 应用敏捷价值观、原则和实践来帮助您的团队获得测试自动化的动力。

Apply agile values, principles, and practices to help your team get traction on test automation.

图像重复任务、持续集成和构建过程、单元测试、功能测试、负载测试和数据创建都是自动化的理想选择。

Repetitive tasks, continuous integration and build processes, unit tests, functional tests, load tests, and data creation are all good candidates for automation.

图像象限 3 测试,如可用性测试和探索性测试,可能会受益于一些自动化来设置测试场景和分析结果,但人类的本能、批判性思维和观察无法自动化。

Quadrant 3 tests such as usability testing and exploratory testing may benefit from some automation to set up test scenarios and analyze results, but human instincts, critical thinking, and observation can’t be automated.

图像一个简单的、全团队的方法,使用迭代反馈,并花足够的时间可以帮助你开始一个好的解决方案。

A simple, whole-team approach, using iterative feedback, and taking enough time can help you get started on a good solution.

图像在制定自动化策略时,从最痛苦的地方开始,考虑多层次的方法,并努力不断地重新审视和改进你的策略,而不是从一开始就达到完美。

When developing an automation strategy, start with the greatest area of pain, consider a multi-layered approach, and strive for continuously revisiting and improving your strategy rather than achieving perfection from the start.

图像在决定自动化什么时考虑风险和投资回报率。

Consider risk and ROI when deciding what to automate.

图像花时间边做边学;将敏捷编码实践应用于测试。

Take time to learn by doing; apply agile coding practices to tests.

图像决定是否可以简单地在内存中构建输入,或者是否需要数据库中的生产型数据。

Decide whether you can simply build inputs in-memory, or whether you need production-style data in a database.

图像提供测试数据,使测试独立、可重新运行并尽可能快。

Supply test data that will allow tests to be independent, rerunnable, and as fast as possible.

图像一次满足一种工具需求,确定您的需求,然后决定选择或构建哪种类型的工具来满足您的需求。

Take on one tool need at a time, identify your requirements, and decide what type of tool to choose or build that fits your needs.

图像使用良好的测试自动化开发实践,并花时间进行良好的测试设计。

Use good development practices for test automation, and take time for good test design.

图像自动化工具需要适应团队的开发基础设施。

Automated tools need to fit into the team’s development infrastructure.

图像版本控制自动化测试以及他们验证的生产代码。

Version-control automated tests along with the production code that they verify.

图像良好的测试管理确保测试可以提供系统和开发进度的有效文档。

Good test management ensures that tests can provide effective documentation of the system and of development progress.

图像立即开始测试自动化。

Get started on test automation today.

第五部分 测试人员生命中的迭代

Part V An Iteration in the Life of a Tester

每当我们与敏捷开发相对较新的参与者进行教程、网络研讨会或问答环节时,我们总是会被问到诸如“在迭代的第一部分,在任何东西准备好测试之前,测试人员做什么?”这样的问题。或“用户验收测试在何处适合敏捷发布周期?” 在敏捷过程中,很容易阐述谁应该做什么以及何时做的理论,但我们发现根据自己的经验给出具体的例子是我们可以给敏捷新手的最好帮助。通过与许多不同的敏捷团队的交谈,我们了解到在适用于敏捷开发和测试的方面存在很多共性。

Whenever we do tutorials, webinars, or Q&A sessions with participants who are relatively new to agile development, we’re always asked questions such as “What do testers do during the first part of an iteration before anything’s ready to test?” or “Where does user acceptance testing fit into an agile release cycle?” It’s easy to expound on theories of who should do what and when, in an agile process, but we find giving concrete examples from our own experience is the best help we can give agile newbies. Through our talking to many different agile teams, we’ve learned that there’s a lot of commonality in what works well for agile development and testing.

在本书的这一部分,我们将在整个迭代过程中跟踪敏捷测试人员的生活。实际上,我们将探索的不仅仅是一次迭代。我们将从测试人员在发布或主题规划期间所做的事情开始,当团队查看它将为几个即将到来的迭代所做的工作时。我们将举例说明测试人员可以做些什么来帮助团队成员在开始迭代时迅速上手。我们将展示编码和测试如何成为交付软件的一个集成过程的一部分,并且我们将描述测试人员和程序员如何紧密和渐进地合作。我们将解释测试人员可以帮助他们的团队保持正轨和衡量进度的不同方式,包括衡量指标和处理缺陷的有用方法。我们将研究与测试相关的活动,这些活动涉及结束迭代并寻找改进下一个迭代的方法。

In this part of the book, we’ll follow an agile tester’s life throughout an iteration. Actually, we’ll explore more than just an iteration. We’ll start with what testers do during release or theme planning, when the team looks at the work it will do for several upcoming iterations. We’ll give examples of what testers can do to help the team members hit the ground running when they start the iteration. We’ll show how coding and testing are part of one integrated process of delivering software, and we’ll describe how testers and programmers work closely and incrementally. We’ll explain different ways that testers can help their teams stay on track and gauge progress, including useful approaches to metrics and handling defects. We’ll look at testing-related activities involved in wrapping up an iteration and finding ways to improve for the next one. Finally, we’ll examine a tester’s role in a successful release, including the end game, UAT, packaging, documentation, and training.

敏捷测试生活片段中描述的活动可以由团队中的任何人执行,而不仅仅是测试专家。在某些团队中,所有团队成员都可以并且确实执行任何任务,无论是开发、测试、数据库、基础设施还是其他任务。为简单起见,在本节中,我们假设我们正在跟踪主要角色是测试的人,因为他们帮助交付高质量的软件。

The activities described in this slice-of-life look at agile testing can be performed by anyone on the team, not only testing specialists. On some teams, all team members can, and do, perform any task, be it development, testing, database, infrastructure, or other tasks. For simplicity, in this section we’ll assume we’re following someone whose primary role is testing as they help to deliver high-quality software.

第 15 章发布或主题规划中的测试人员活动

Chapter 15 Tester Activities in Release or Theme Planning

图像

敏捷开发团队在每次迭代中完成故事并交付生产就绪软件,但提前规划大局或更大的功能块。一个主题、史诗或项目可能包含多个迭代。在本章中,我们将了解当团队花时间计划发布时测试人员会做什么。我们还考虑了一些方法来跟踪我们的开发是否按预期进行,或者是否需要更正路线。

Agile development teams complete stories and deliver production-ready software in every iteration but plan the big picture or a larger chunk of functionality in advance. A theme, epic, or project may encompass several iterations. In this chapter, we look at what testers do when their team takes time to plan their release. We also consider ways to track whether our development is proceeding as anticipated, or if course corrections are needed.

发布计划的目的

The Purpose of Release Planning

软件团队尝试敏捷开发的原因之一是因为他们知道长期计划行不通。大多数业务环境都是多变的,优先级每周甚至每天都在变化。敏捷开发应该避免“预先进行大设计”。我们中的大多数人都经历过制定计划,结果证明是在浪费精力。但是我们必须对我们的客户正在寻找什么以及我们如何交付它有一些了解,以便有一个良好的开端。幸运的是,敏捷方法可以使计划成为一种有用的方法,让我们在了解如何交付产品方面抢先一步。

One reason software teams try agile development is because they know long-range plans don’t work. Most business environments are volatile, and priorities change every week, or even every day. Agile development is supposed to avoid “big design up front.” Most of us have experienced making plans that turned out to be a waste of effort. But we have to have some understanding of what our customer is looking for and how we might deliver it in order to get off to a good start. Fortunately, an agile approach can make planning a useful way to give us a head start on knowing how we will deliver the product.

XP 团队可能每隔几个月就花一天时间进行发布计划。其他敏捷团队在准备开始一个主题、史诗或主要功能时会提前计划,我们将其视为一组相关的故事。他们努力理解主题或高水平发布。客户对我们应该提供什么的看法是什么?发布的目的是什么?大局是什么?它将为企业和客户带来什么价值?还涉及哪些其他团队或项目并需要协调?UAT什么时候开始?什么时候将代码发布到暂存区、生产区?如果我们走上正轨,我们需要知道哪些指标?这些一般问题在发布计划中得到解决。

XP teams may take a day every few months for release planning. Other agile teams do advance planning when getting ready to start on a theme, epic, or major feature, which we think of as a related group of stories. They work to understand the theme or release at a high level. What is the customer’s vision of what we should be delivering? What’s the purpose of the release? What’s the big picture? What value will it deliver to the business, to the customers? What other teams or projects are involved and require coordination? When will UAT take place? When will code be released to staging, to production? What metrics do we need to know if we’re on track? These general questions are addressed in release planning.

有些团队不会花太多时间做发布计划活动。优先级变化很快,即使在特定的功能主题内也是如此。没有人愿意预先做太多最终被浪费的工作。一些团队只查看前几个故事,以确保他们可以顺利开始。至少,团队希望了解足够多的信息,以便让他们的系统架构指向正确的方向,并开始处理前几个故事。

Some teams don’t spend much time doing release planning activities. Priorities change quickly, even within a particular theme of features. Nobody wants to do too much work up front that ends up being wasted. Some teams just look at the first couple of stories to make sure they can get a running start. At the very least, teams want to know enough to get their system architecture pointed in the right direction and get started on the first few stories.

这些计划会议并非旨在详细计划发布的每个迭代。而且我们知道我们无法准确预测每次迭代可以完成多少个故事。但是,我们确实知道我们的平均速度,因此我们可以大致了解发布的可能范围。团队讨论功能和故事,试图从 20,000 英尺的高度了解发布内容以及可能需要多少次迭代才能完成。我们都喜欢 Mike Cohn 在他的《敏捷估算和规划》一书中提出的发布规划方法[2005]。企业想要包含的故事是相对于彼此的大小,然后根据它们提供的价值对功能进行优先级排序。团队可以通过功能识别“薄片”以确定哪些故事绝对必须完成,哪些在范围内,哪些“最好有”可以推迟到以后。他们着眼于故事之间的依赖关系、相对风险和其他决定特性编码顺序的因素。故事编码的顺序与故事的大小一样重要,有时甚至更重要。团队希望在发布的第一次迭代中交付价值。

These planning meetings aren’t intended to plan every iteration of the release in detail. And we know we can’t predict exactly how many stories we can complete each iteration. However, we do have an idea of our average velocity, so we can get a general idea of the possible scope of the release. The team talks about the features and stories, trying to get a 20,000-foot view of what can go into the release and how many iterations it might take to complete. Both of us like Mike Cohn’s approach to release planning in his book Agile Estimating and Planning [2005]. Stories that the business wants to include are sized relative to each other, and then features are prioritized according to the value they deliver. The team may identify “thin slices” through the features to determine what stories absolutely have to be done, what’s in scope, what “nice-to-haves” could be put off until later. They look at dependencies between stories, relative risk, and other factors that determine the order in which features should be coded. The order in which stories are coded is as important, or sometimes more important, than the size of the stories. Teams want to deliver value the first iteration of the release.

发布计划让开发人员和客户有机会考虑计划的功能对更大系统的影响,澄清假设,并查看可能影响哪些故事首先完成的依赖关系。他们可能会考虑更高级别的测试,以及是否需要新的资源,例如测试环境和软件。

Release planning is a chance for the developers and customers to consider the impact of the planned features on the larger system, clarify assumptions, and look at dependencies that might affect what stories are done first. They may think about testing at a high level and whether new resources such as test environments and software will be needed.

让我们跟随我们的敏捷测试员完成发布计划活动,看看她如何通过她独特的视角和专注来贡献价值。

Let’s follow our agile tester through release planning activities and see how she contributes value through her unique perspective and focus.

浆纱

Sizing

敏捷团队估计每个故事的相对大小。一些团队随着他们的发展而调整规模,将估计推迟到他们实际完成故事的迭代。其他人甚至在发布计划之前召开会议来评估故事。一些开发人员和客户团队坐在一起,同时编写和估算故事的大小。规模调整的目的是让程序员让企业了解每个故事的成本,并帮助他们确定优先级并计划前几次迭代。合作多年的高效团队可能会采取不太正式的方法。对于新的敏捷团队,学习调整故事规模需要大量的实践和经验。正确调整每个故事的大小并不重要,但要足够接近以使客户了解故事的大小,以便他们可以根据更好的信息确定优先级。随着时间的推移,单个故事大小的变化将被平均化,我们发现一个主题或相关的一组故事所花费的时间与预期的差不多。

Agile teams estimate the relative size of each story. Some teams size as they go, delaying the estimation until the iteration where they’ll actually complete the story. Others have meetings to estimate stories even in advance of release planning. Some developer and customer teams sit together to write and estimate the size of stories all at one time. The goal of sizing is for the programmers to give the business an idea of the cost of each story and to help them prioritize and plan the first few iterations. High-functioning teams who’ve worked together for years may take a less formal approach. For new agile teams, learning to size stories takes a lot of practice and experience. It’s not important to get each story sized correctly but to be close enough to give customers some idea of how big the stories are so they can prioritize with better information. Over time, variations on individual story sizing will average out, and we find that a theme or related group of stories takes about the amount of time expected.

如何调整故事大小

How to Size Stories

至于如何计算故事大小,不同的团队使用不同的技术,但同样,我们喜欢 Mike Cohn 确定故事大小的方法。我们根据故事点、理想的日子或简单的“小、中、大”来衡量。每个故事相对于其他故事的大小是重要因素。例如,向现有用户界面添加一个输入字段显然比从头开发一个全新的屏幕要小得多。

As far as how to calculate story size, different teams use different techniques, but again, we like Mike Cohn’s approach to determining story size. We size in story points, ideal days, or simply “small, medium, large.” The relative size of each story to others is the important factor. For example, adding an input field to an existing user interface is obviously much smaller than developing a brand new screen from scratch.

如果企业知道平均速度(团队每次迭代完成的故事点数)并且对每个想要完成的故事有初始规模估计,它就会知道实施给定主题可能需要多长时间。与任何其他开发方法一样,没有任何保证,因为估计就是这样。尽管如此,企业仍可以很好地计划以开展其日常活动。

If the business knows the average velocity (the number of story points the team completes each iteration) and has the initial size estimates of each story it wants to get done, it has an idea of how long it might take to implement a given theme. As with any other development methodology, there are no guarantees, because estimates are just that. Still, the business can plan well enough to conduct its usual activities.

我们的团队使用计划扑克(在 Mike Cohn 的《敏捷估算和规划》一书中有解释)来估算故事规模。在计划扑克中,每个团队成员都有一副牌。每张卡片上都有许多点。该过程从客户或产品所有者阅读故事并解释其目的和将提供的价值开始。他可能会列出一些满意的条件或高级测试用例。经过简短的讨论后,团队成员每人举起一张积分卡,代表他们认为从他们的角度来看这个故事有多“大”。他们讨论点值上的任何重大差异并再次估算,直到达成共识。图 15-1显示了团队成员正在谈论他们各自刚刚显示的点值。这需要一个快速的过程——关于细节的长时间讨论不会导致更准确的尺寸估计。

Our teams use planning poker (explained in Mike Cohn’s book Agile Estimating and Planning) to estimate story size. In planning poker, each team member has a deck of cards. Each card has a number of points on it. The process begins with the customer or product owner reading a story and explaining its purpose and the value it will deliver. He might list a few conditions of satisfaction or high-level test cases. After a brief discussion, team members each hold up a point card that represents how “big” they think the story is from their perspective. They discuss any big differences in point value and estimate again until they reach consensus. Figure 15-1 shows team members talking about the point values they each just displayed. This needs to be a quick process—long discussions about details don’t result in more accurate size estimates.

图 15-1计划扑克。经 Mike Thomas 许可使用。版权所有 2008。

Figure 15-1 Planning poker. Used with permission of Mike Thomas. Copyright 2008.

图像

一些团队根据在规定时间内完成给定故事需要多少人来计算故事的相对大小。其他人估计一个人需要多少天才能完成它。使用一种对所有团队成员都有意义的测量方法,并提供估计之间的一致性。

Some teams figure the relative sizes of stories by how many people are needed to complete a given story in a set amount of time. Others estimate how many ideal days one person would need to finish it. Use a measurement that makes sense to all team members and one that provides consistency among estimates.

测试人员在调整故事中的作用

The Tester’s Role in Sizing Stories

我们最喜欢的说法之一是,“没有测试就没有故事”。但是,我们遇到过一些团队,其中测试未包含在故事大小的估计中。在某些情况下,测试一项功能可能比编写代码花费更长的时间。

One of our favorite sayings is, “No story is done until it’s tested.” However, we’ve run across teams where testing wasn’t included in estimates of story size. In some cases, testing a piece of functionality might take longer than coding it.

根据我们的经验,测试人员通常与其他团队成员有不同的观点。他们通常对领域有广泛的了解,并且可以快速识别一个故事可能对系统其余部分产生的“连锁反应”。他们还倾向于考虑与可能需要完成的开发没有直接关系的活动,例如在新的或更改的界面上培训用户。

In our experience, testers usually have a different viewpoint than other team members. They often have a broad understanding of the domain and can quickly identify “ripple effects” that one story might have on the rest of the system. They also tend to think of activities not directly related to development that might need to be done, such as training users on a new or changed interface.

丽莎的故事

Lisa’s Story

测试人员在故事大小调整过程中做什么?我从几个角度快速思考这个故事。故事要解决什么业务问题,或者它提供什么业务价值?如果不清楚,我会向产品负责人提问。最终用户将如何实际使用该功能?如果还是不清楚,我请产品负责人举个简单的例子。我可能会问,“它可能出错的最糟糕的事情是什么?” 这种消极的方法有助于衡量故事的风险。哪些测试注意事项可能会影响故事的大小?如果很难获得测试数据或故事涉及第三方,则测试可能比编码花费更长的时间。我试图快速排除任何隐藏的假设。是否存在依赖关系或特殊的安全风险?应用程序的这一部分是否需要处理大负载?

What does a tester do during the story sizing process? I think quickly about the story from several viewpoints. What business problem is the story solving, or what business value does it deliver? If this isn’t clear, I ask the product owner questions. How will the end user actually use the feature? If it’s still not clear, I ask the product owner for a quick example. I might ask, “What’s the worst thing that could go wrong with it?” This negative approach helps gauge the story’s risk. What testing considerations might affect the story’s size? If test data will be hard to obtain or the story involves a third party, testing might take longer than coding. I try to quickly flush out any hidden assumptions. Are there dependencies or special security risks? Will this part of the application need to handle a big load?

许多故事都不够大,不足以保证那么多思考。通常,我们不需要太多细节来了解相对大小。然而,如果一个故事被低估了五到十倍,你的团队真的会被烧毁。我们曾经对一个最终至少是十倍大小的故事给出了一个相对较小的估计。这些是我们想通过提出好的问题来避免的灾难。

Many stories aren’t big enough to warrant that much thought. Usually, we don’t need much detail to get an idea of relative size. However, your team can really get burned if a story is underestimated by a factor of five or ten. We once gave a relatively small estimate to a story that ended up being at least ten times the size. These are the disasters we want to avoid by asking good questions.

——丽莎

—Lisa

测试人员需要参与规模调整过程。一些团队认为只有程序员应该参与,但当测试人员积极参与时,他们可以帮助获得更准确的故事规模,这符合整个团队的最大利益。

Testers need to be part of the sizing process. Some teams think that only programmers should participate, but when testers are active participants, they can help to get a much more accurate story sizing, which is in the best interests of the whole team.

规模故事的一个例子

An Example of Sizing Stories

让我们假设我们有图 15-2中的故事来评估。

Let’s imagine we have the story in Figure 15-2 to size up.

图 15-2 Story 删除项目

Figure 15-2 Story to delete items

图像

产品负责人阅读故事后,会发生以下讨论:

After the product owner reads the story, the following discussion ensues:

产品负责人: “我们只是想为用户提供一些简单的方法来删除项目,但我们并没有考虑具体的实现方式。”

Product Owner: “We just want some easy way for users to delete items, but we don’t have a specific implementation in mind.”

测试人员: “他们应该能够一次删除多个项目吗?”

Tester: “Should they be able to delete several items at once?”

产品负责人: “哦,是的,让它尽可能简单。”

Product Owner: “Oh, yes, just make it as easy as possible.”

测试员: “如果他们不小心删除了他们想购买的商品怎么办?”

Tester: “What if they accidentally delete an item they wanted to buy?”

产品负责人: “有没有什么办法可以保存已删除的项目以供以后检索?”

Product Owner: “Is there some way the deleted items can be saved for later retrieval?”

程序员: “当然可以,但是你应该为此写一个新故事。现在,我们应该从基本的删除功能开始。”

Programmer: “Sure, but you should write a new story for that. For now, we should start with the basic delete functionality.”

测试人员: “上一个版本我们实现了愿望清单功能。您是否希望用户能够将商品从他们的购物篮移动到他们的愿望清单?那也将是一个新的故事。”

Tester: “Last release we implemented a wish list feature. Do you want users to be able to move items from their shopping basket to their wish list? That would be a new story also.”

产品负责人: “是的,那是我们想做的另外两个故事,这是肯定的。我会把它们写下来,我们也可以调整它们的大小。但如果必须的话,我们绝对可以将它们推迟到下一个版本。”

Product Owner: “Yes, those are two more stories we want to do, for sure. I’ll write those down, we can size them also. But we could definitely put them off until the next release, if we have to.”

测试人员: “此功能可能发生的最糟糕的事情是什么?”

Tester: “What’s the worst thing that could happen with this feature?”

产品负责人: “如果他们不知道如何删除,他们可能会放弃整个购物篮。它必须非常简单明了。”

Product Owner: “If they can’t figure out how to delete, they might just abandon their whole shopping basket. It has to be really easy and obvious.”

ScrumMaster 要求进行估算。该团队了解他们只是在调整删除项目的基本故事的大小,而不是为了对已删除的项目做其他事情。他们很快就点值达成一致。

The ScrumMaster calls for an estimate. The team understands they’re sizing only the basic story for deleting items, not for doing something else with the deleted items. They quickly agree on a point value.

我们再看一个故事。(见图15-3。)

Let’s look at another story. (See Figure 15-3.)

图 15-3关于运输速度的故事

Figure 15-3 Story on shipping speed

图像

测试人员: “用户可以选择的运输速度是多少?”

Tester: “What are the shipping speeds the user can choose?”

产品负责人: “标准的 5 天、2 天和次日。”

Product Owner: “Standard 5-day, 2-day, and next-day.”

程序员: “我们或许应该从只提供一种速度开始,然后计算该成本。然后我们可以轻松实现其他两种速度。”

Programmer: “We should probably start by only offering one speed, and calculating that cost. Then we can easily implement the other two speeds.”

Product Owner: “就这样分解就好了。”

Product Owner: “It’s fine to break it up like that.”

测试人员: “我们会使用 BigExpressShipping 的 API 来根据重量和目的地计算费用吗?”

Tester: “Will we use BigExpressShipping’s API to calculate cost based on weight and destination?”

程序员: “那是最简单的。”

Programmer: “That would be the easiest.”

队员们举起他们的积分卡。测试人员和其中一名程序员举起 8;其他开发人员举起 5。

The team holds up their point cards. The tester and one of the programmers hold up an 8; the other developers hold up a 5.

ScrumMaster: “你们两个为什么选择8?”

ScrumMaster: “Why did you two choose 8?”

测试人员: “我们之前从未使用过 BigExpressShipping 的成本 API,我不确定这将如何影响我们的测试。我们必须找出如何访问他们的系统进行测试。”

Tester: “We’ve never used BigExpressShipping’s cost API before, and I’m not sure how that will impact our testing. We have to find out how to access their system for testing.”

其他 8 分的程序员: “我同意,我认为测试工作比这个故事的编码工作更紧张。”

Other Programmer with 8: “I agree, I think the testing effort is more intense than the coding effort for this story.”

团队同意将故事的大小分为八个要点。

The team agrees to size the story as eight points.

这个调整过程可能发生在计划会议之前,如果很久以前就对故事进行了调整或估算,团队可能希望确保他们对故事的大小感到满意。团队可能已经改变或可能更有经验。这些因素中的任何一个都可以使团队改变估计。

This sizing process may occur before the planning meeting, and if the stories were sized or estimated a long time ago, the team might want to make sure they feel comfortable with the story sizes. Teams may have changed or may be more experienced. Either of those factors can make a team change the estimates.

很多时候,一个故事会有很大的测试部分,而编码工作量很小。在其他时候,情况正好相反。考虑所有观点很重要。

There are many times when a story will have a large testing component, and the coding effort is small. At other times, the reverse will be true. It’s important to consider all perspectives.

丽莎的故事

Lisa’s Story

我们的团队开始害怕故事大小会议,因为我们对细节的讨论时间过长,而且会议总是比预定时间持续很长时间。从那时起,我们的 ScrumMaster 找到了让我们走上正轨的方法。她使用一个鸡蛋计时器来计时讨论,并在每次沙子用完时停止讨论,看看我们是否认为我们真的需要更多时间来提问。我们的产品负责人也了解了我们进行估算所需的信息,并且通常拥有我们需要的信息。我们还学会了只处理可能在接下来的几次迭代中出现的故事。

Our team grew to dread story sizing meetings, because we got into overly long discussions about details, and the meetings always lasted long past the scheduled time. Since then, our ScrumMaster has found ways to keep us on track. She uses an egg timer to time discussions, and stops them each time the sand runs out to see if we think we really need more time to ask questions. Our product owner has also learned what information we need for estimating and usually has what we need. We also learned to only work on stories that were likely to come up in the next few iterations.

在我们所有的会议中,小传统已经发展起来,使会议变得更加有趣。总会有人带着零食来参加迭代计划会议。在站立会议中,我们传递一个笔形电筒和激光笔的组合,所以我们每个人都拿着它来报告我们正在做的事情。我们总是以比赛结束故事规模会议,看看谁能将他或她的一副计划扑克牌扔进他们住的小塑料桶里。图 15-4显示了这个愚蠢但有趣的会议结束活动。永远记住享受的敏捷价值,并在会议中享受一些乐趣。

With all of our meetings, little traditions have grown to make the meetings more fun. Someone always brings treats to iteration planning meetings. In stand-up meetings, we pass around a combination penlight and laser pointer, so each of us holds it as we report on what we’re working on. We always end story sizing meetings with a competition to see who can throw his or her deck of planning poker cards into the small plastic tub where they live. Figure 15-4 shows this goofy but fun meeting-ending activity. Always remember the agile value of enjoyment and have some fun with your meetings.

——丽莎

—Lisa

图 15-4会议结束的传统。经 Mike Thomas 许可使用。版权所有 2008。

Figure 15-4 A meeting-ending tradition. Used with permission of Mike Thomas. Copyright 2008.

图像

优先排序

Prioritizing

发布计划会议的目的还在于了解团队将尝试在发布日期之前完成哪些故事。客户对故事进行优先级排序,但可能存在依赖关系,因此首先处理某些故事是有意义的,即使它们不是最高优先级。重要的是,团队了解并非所有故事都能在发布日期之前完成的可能性。敏捷的基本前提之一是交付可工作的软件,因此首先完成最高价值的故事非常重要,这样我们交付的软件才能满足客户的需求。

The purpose of the release planning meeting is also to get an idea of what stories the team will try to finish by the release date. The customers prioritize the stories, but there may be dependencies, so it makes sense to do certain stories first, even if they aren’t the highest priority. It is important that the team understands the possibility that not all of the stories will get completed by the release date. One of the basic premises of agile is to deliver working software, so it is important to have the highest-value stories completed first so that the software we do deliver meets the customer’s needs.

为什么我们优先考虑故事

Why We Prioritize Stories

每个人的目标都是在每次迭代中交付真正的价值。测试人员可以帮助团队挑选出必须工作的核心功能。在第 8 章中,我们解释了“薄片”或“钢丝”概念,确定了通过功能首先进行编码和测试的路径,并在第一个关键路径起作用后添加更多功能。这个概念也适用于发布级别。故事的顺序很重要。Lisa 的团队有时会分解一个故事并抽出一个功能的核心部分在第一次迭代中完成。

Everyone’s goal is to deliver real value in each iteration. Testers can help the team pick out the core functionality that has to work. In Chapter 8, we explained the “thin slice” or “steel thread” concept, identifying one path through the functionality to code and test first, and adding more features after the first critical path works. This concept applies at the release level, too. The order of the stories is critical. Lisa’s team will sometimes break up a story and pull out a core part of a feature to do in the first iteration.

一些没有进行全面发布计划的团队确实会花时间查看故事并决定应该首先发布哪两三个。这样,他们在发布的第一次迭代中就提供了商业价值。

Some teams that don’t do full-blown release planning do take time to look at the stories and decide which two or three should be first. That way, they deliver business value in the very first iteration of the release.

让我们看一个例子。

Let’s look at an example.

如果我们的主题是为在线购物者提供选择运输选项的能力,然后根据重量、运输速度和目的地计算运输成本,那么完成简单的故事甚至故事的子集可能是个好主意,以便结账过程可以端到端地进行。首先只允许标准的 5 天送达、重量小于 10 磅的物品以及目的地在美国大陆。当用户可以获得该场景的运费并结账时,团队可以决定下一个优先级。它们可能包括重量级物品、更快的运输速度、运送到夏威夷和阿拉斯加以及运送到加拿大和墨西哥。

If our theme is providing the ability for an online shopper to choose shipping options and then calculate the shipping cost based on weight, shipping speed, and destination, it may be a good idea to complete simple stories or even subsets of stories so that the checkout process can proceed end-to-end. Start by only allowing standard 5-day shipping, items less than 10 pounds, and destinations in the continental United States. When the user can get the shipping cost for that scenario and check out, the team can decide the next priorities. They may include heavyweight items, faster shipping speeds, shipping to Hawaii and Alaska, and shipping to Canada and Mexico.

通过首先提供这个薄片,测试人员可以立即开始测试。程序员还测试了他们的设计和代码集成步骤,因此对整个功能完成后的工作方式有一个清晰的认识。

By providing this thin slice first, the testers have something to start testing immediately. The programmers have also tested their design and code integration steps and so have a solid idea of how things will work when the whole feature is complete.

确定优先级时的测试注意事项

Testing Considerations While Prioritizing

团队了解大局或主题很重要。在我们的示例中,团队成员知道在美国大陆以外运输的故事将在稍后出现。这些知识可能会影响他们如何实施第一个故事。这并不意味着他们必须为每一种可能发生的情况做好计划,但如果他们知道他们需要更多的运输选项,他们可能会实施下拉列表而不是基本的文本字段。无需进行不必要的工作或返工。

It is important that the team understands the big picture or theme. In our example, team members know the stories for shipping outside the continental United States will come later. This knowledge may affect how they implement the first story. This doesn’t mean they have to plan for every eventuality, but if they know they need more shipping options, they may implement a drop-down list rather than a basic text field. No need to make more work or rework than necessary.

在发布计划期间,我们还考虑了故事的相对风险。如果某些故事有很多未知数,最好将它们包括在早期迭代中,这样如果故事“爆炸”并且花费的时间比估计的多得多,就有时间恢复。这同样适用于一个故事,如果没有完成或实施不当,将会产生代价高昂的负面影响。尽早安排将留出更多时间进行测试。

During release planning, we also consider the relative risk of the stories. If certain stories have many unknowns, it might be best to include them in an early iteration, so there’s time to recover if a story “blows up” and takes much more time than estimated. The same may apply to a story which, if not completed or implemented incorrectly, would have a costly negative impact. Scheduling it early will leave more time for testing.

如果需要新技术或软件,最好通过开发一个简单的故事来学习它,并为以后的迭代计划更困难的故事。这项新技术可能会也可能不会影响您的测试自动化。您可能需要更多时间来检查影响。如果这些功能都是全新的,并且团队需要更多时间来了解它们应该如何工作,那么计划在第一次迭代中以低于平均速度的速度进行。这样,您将有更多时间编写能够正确指导开发的测试。从测试角度和开发角度识别风险并确定哪种方法最有意义。这是让整个团队参与规划会议的重要原因之一。

If new technology or software is needed, it might be good to learn it by developing a straightforward story and plan more difficult ones for later iterations. This new technology may or may not affect your test automation. You may want more time to check out the impact. If the features are all brand new and the team needs more time to understand how they should work, plan to do less than your average velocity for the first iteration. That way, you’ll have more time to write tests that will correctly guide development. Identify risks and decide what approach makes the most sense from a testing perspective as well as a development perspective. This is one of the reasons it is important to include the whole team in the planning sessions.

从测试的角度来看故事是必不可少的。这是测试人员最有价值的地方。团队需要以可测试的小块进行开发,以帮助决定为哪个迭代暂定计划哪些故事。这里的关键是可测试的。许多新的敏捷团队认为小块意味着首先完成所有的数据库工作,或者所有的配置工作。可测试并不一定意味着它也需要 GUI。例如,计算运费的算法是一段独立的代码,可以独立于任何用户界面进行测试,但需要进行大量测试。这可能是第一次迭代的好故事。它可以作为独立代码进行测试,然后结合 UI 和系统的其他部分进行测试。

Looking at the stories from a testing viewpoint is essential. This is where testers add the most value. The team needs to develop in small, testable chunks in order to help decide what stories are tentatively planned for which iteration. The key here is testable. Many new agile teams think small chunks means doing all of the database work first, or all of the configuration stuff. Testable doesn’t necessarily mean it needs a GUI either. For example, the algorithm that calculates shipping cost is an independent piece of code that can be tested independently of any user interface but requires extensive testing. That might be a good story for the first iteration. It can be tested as free-standing code and then later tested in combination with the UI and other parts of the system.

测试人员可能会游说通过代码快速获得端到端的跟踪子弹,这样他们就可以构建一个自动化框架,然后充实它随着故事发展的进行。如果有故事提出了巨大的测试挑战,那么尽早完成这些故事可能会很好。例如,如果该版本包括实施新的第三方工具以从模板和动态数据创建文档,则有许多排列需要测试。如果团队不熟悉该工具,测试人员可以要求团队考虑在发布的第一次迭代中做那些故事。

The testers may lobby for getting an end-to-end tracer bullet through the code quickly, so they can build an automation framework, and then flesh it out as the story development proceeds. If there are stories that present a big testing challenge, it might be good to do those early on. For example, if the release includes implementing a new third-party tool to create documents from templates and dynamic data, there are many permutations to test. If the team is unfamiliar with the tool, the testers can ask the team to consider doing those stories in the first iteration of the release.

范围内有什么?

What’s in Scope?

敏捷团队不断管理范围,以便在保持质量的同时满足业务期限。高价值的故事是第一要务。“锦上添花”的故事可能会被排除在发行版之外。

Agile teams continually manage scope in order to meet business deadlines while preserving quality. High-value stories are the first priority. Stories that are “nice-to-haves” might be elbowed out of the release.

丽莎的故事

Lisa’s Story

我们团队的客户按优先顺序列出他们的故事,然后在发布之前必须完成的故事与可以安全推迟的故事之间划清界线。他们将不太重要的故事称为“线下”,而这些故事可能永远也做不完。

Our team’s customers list their stories in priority order and then draw a line between the stories that must be done before the release can occur, and the ones that could safely be put off. They call the less important stories “below the line,” and those stories may never get done.

例如,当我们提出允许退休计划参与者从他们的退休账户借钱的主题时,有一个“线下”故事向任何贷款状态变为“待违约”或“违约”的参与者发送电子邮件。 ” 当贷款处于“违约”状态时,借款人必须为余额缴纳税款和罚款。电子邮件对借款人非常有帮助,但对我们的业务而言,它并不像请求、批准和分配贷款或处理贷款支付的软件那么重要。

For example, when we undertook the theme to allow retirement plan participants to borrow money from their retirement accounts, there was a “below the line” story to send emails to any participants whose loans are changing status to “pending default” or “default.” When the loan is in “default” status, the borrower must pay taxes and penalties on the balance. The email would be extremely helpful to the borrowers, but it wasn’t as important to our business as the software to request, approve and distribute loans, or process loan payments.

电子邮件的故事没有进入发布。直到两年多后,在不知道自己的贷款将违约的人们发出足够多的抱怨之后,才完成这项工作,直到为时已晚。

The email story didn’t make it into the release. It wasn’t done until more than two years later, after enough complaints from people who didn’t know their loans were going into default until it was too late.

——丽莎

—Lisa

Janet 与一个团队合作,该团队的客户错误地假设所有功能都会进入他们的版本,并且当他们确定优先级时,他们只是选择先完成哪些故事。当团队的其他成员意识到误解时,他们也实施了线上线下故事的想法。它有助于跟踪进度并使放在线下的故事非常明显。

Janet worked with a team whose customers were under the misplaced assumption that all of the features would get into their release and that when they were prioritizing, they were just picking which stories got done first. When the rest of the team realized the misunderstanding, they also implemented the idea of stories above and below the line. It helped to track progress as well as make the stories that were dropped below the line very visible.

截止日期和时间表

Deadlines and Timelines

许多领域围绕日历上的固定日期展开。零售企业在假期赚取大部分利润。一个 Internet 零售站点在 10 月 1 日之前实现所有新功能是明智的。临近购买高峰期的新特性是有风险的。丽莎公司的客户必须在一年中的特定时期完成政府要求的任务。如果某项功能今年发布为时已晚,则通常会推迟到明年发布,因为必须解决更紧迫的优先事项。监管变化有具体的时间表,组织对时间表没有选择。

Many domains revolve around fixed dates on the calendar. Retail businesses make most of their profit during the holiday season. An Internet retail site is smart to have all new features implemented by October 1. Implementing a new feature close to the peak buying period is risky. Lisa’s company’s customers must complete government-required tasks during certain periods of the year. When it’s too late for a feature to get released this year, it often gets put off for the next year, because more urgent priorities must be addressed. Regulatory changes have specific timelines, and organizations have no choice about the timeline.

珍妮特的故事

Janet’s Story

在编写本书时,我正计划与我在 WestJet 的团队一起发布。我们有几个可能的故事,并与客户一起决定发布的样子。我们进行了一项法规变更,对程序员来说工作很轻松,但对测试人员来说却很繁重。它需要在特定日期之前投入生产,因此我们考虑发布的其他故事也考虑到了这一点。

While working on this book, I was planning a release with my team at WestJet. We had several possible stories and worked with the customers to decide what the release would look like. We had one regulatory change that was light work for the programmers, but heavy for the testers. It needed to be in production by a certain date, so the other stories we were considering for the release took that into consideration.

我们决定创建一个只包含一个主要功能的小型维护版本,以及积压工作中的一些错误,这样就不会危及监管变更的发布。当测试人员完成他们的测试时,团队的其他成员开始为下一个版本编写一些详细的故事。

We decided to create a small maintenance release with just that one major feature, along with a few bugs from the backlog so the release of the regulatory change would not be jeopardized. While the testers completed their testing, the rest of the team started some of elaboration stories for the next release.

一个替代计划可能是程序员参与并帮助测试和适应更多功能。但是,整个团队认为该计划效果最好,风险最小。

An alternative plan could have been that the programmers chip in and help test and fit in more features. However, the whole team decided that this plan would work the best with the least amount of risk.

——珍妮特

—Janet

关注价值

Focus on Value

团队很容易开始讨论一个复杂的故事,而忽视这些功能实际提供的价值。发布计划是时候开始询问有关如何使用这些功能以及它们将提供什么价值的示例和用例。在白板上绘制流程图或示例计算有助于确定核心功能。

It’s rather easy for a team to start discussing a complex story and lose sight of what value the features actually deliver. Release planning is the time to start asking for examples and use cases of how the features will be used, and what value they’ll provide. Drawing flowcharts or sample calculations on the whiteboard can help pinpoint the core functionality.

丽莎的故事

Lisa’s Story

我们的产品负责人写了一个故事来提供警告,如果雇主在参与者已经做出贡献后覆盖参与者有资格为退休账户贡献的日期。

Our product owner wrote a story to provide a warning if an employer overrides the date a participant becomes eligible to contribute to a retirement account after the participant has already made contributions.

该警告需要合并到旧版 UI 代码中,而旧版 UI 代码不容易容纳它。团队讨论了如何实施,但每个选项的成本都相当高。编码不仅会很棘手,而且需要大量时间来充分测试它并更新现有的自动化测试。此功能不会为企业提供太多价值,只是对最终用户有一点帮助。该版本已经非常接近功能限制。

The warning needed to be incorporated into the legacy UI code, which didn’t easily accommodate it. The team discussed how it might be implemented, but every option was fairly costly. Not only would coding be tricky, but a lot of time was needed to test it adequately and update existing automated tests. This feature wouldn’t provide much value to the business, just a bit of help to the end users. The release was already pretty close to the limit on features.

其中一位程序员建议提供一份符合标准的参与者的报告,这样计划管理员就可以简单地打电话给雇主,他们可能会需要更正。报告故事比警告故事小得多,可以很容易地放入发布中,并且可以被客户接受。

One of the programmers suggested providing a report of participants who met the criteria so the plan administrators could simply call the employers who may need to make corrections. The report story was much smaller than the warning story, could easily fit into the release, and was acceptable to the customer.

——丽莎

—Lisa

无法保证这些对给定版本中的内容的初始“猜测”会随着时间的推移而保持不变。这就是为什么客户需要了解他们的优先级,在每次迭代结束时采取检查点,并重新评估剩余故事的优先级。

There is no guarantee that these initial “guesstimates” at what will be in a given release will hold up over time. That is why customers needs to understand their priorities, take checkpoints at the end of every iteration, and reevaluate the priorities of remaining stories.

全系统影响

System-Wide Impact

作为测试人员,我们的工作之一就是牢记大局。敏捷测试人员考虑每个故事如何影响整个系统或我们必须与之合作的其他系统。例如,如果玩具仓库对其库存软件进行了更改,并且新代码存在夸大库存商品数量的错误,则该网站可能会售出比可用的更多的热门新玩偶,让成千上万的孩子失望和他们的父母在圣诞节。当风险很高时,即使在发布计划期间,列出可能受主题或故事组影响的系统区域也是一项值得的练习。

One of our jobs as testers is to keep the big picture in mind. The agile tester thinks about how each story might affect the system as a whole, or other systems that ours has to work with. For example, if the toy warehouse makes a change to its inventory software, and the new code has a bug that overstates the number of items in stock, the website might sell more of the hot new doll than there are available, disappointing thousands of children and their parents at Christmas. When risk is high, listing areas of the system that might be affected for a theme or group of stories might be a worthwhile exercise even during release planning.

我们在第 8 章支持团队的业务面临的测试”中讨论了“涟漪效应” 。

We talked about the “ripple effects” in Chapter 8, “Business Facing Tests that Support the Team.”

我们的系统与合作伙伴或供应商的系统之间的联系点始终值得考虑。如果我们没有正确地将其传达给通过 ftp 文件发送给我们的合作伙伴,即使是对 csv 或 xml 文件格式的微小更改也可能产生巨大影响。意味着需要在发布周期中尽早完成第三方更改的故事,以便让第三方进行必要的更改。

Contact points between our system and that of partners or vendors always merit consideration. Even a minor change to a csv or xml file format could have a huge impact if we don’t communicate it correctly to partners who ftp files to us. Stories that mean changes for third parties need to be done early enough in the release cycle to let the third parties make necessary changes.

图 15-5显示了一个涉及现有系统许多部分的新系统的简化图。可能需要不同的工具来测试集成。

Figure 15-5 shows a simplified diagram of a new system that touches many pieces of the existing system. Different tools might be needed to test the integrations.

图 15-5系统影响

Figure 15-5 System impacts

图像

使用过其他一些系统或了解需要在这些系统上进行哪些测试的测试人员可以提供有关新故事影响的宝贵见解。如果还没有探索影响,故事通常需要推迟到未来发布。现在是回顾之前结局不太好的版本的好时机。

Testers who have worked with some of the other systems or understand what testing needs to happen on those systems can offer valuable insight into the impact of a new story. Often, stories will need to be delayed until a future release if the impact has not been explored. This is a good time to recall previous releases that didn’t end so well.

第三方参与

Third-Party Involvement

在大型项目中与供应商工具、合作伙伴或其他承包商团队合作会使发布计划复杂化。如果您团队以外的任何人负责项目的某些部分,那么这就是您无法控制的一部分。如果您需要与其他人协调,包括系统可能的新用户,最好早点开始。

Working with vendor tools, partners, or other contractor teams on a big project complicates release planning. If anyone outside your team is responsible for some part of the project, that’s one piece that’s out of your control. If you need to coordinate with others, including possible new users of the system, it’s best to start early.

Lisa 的团队编写了多个接口,允许用户将数据上传到他们的系统。在每种情况下,他们都必须尽早将建议的文件格式提供给用户,以确保它适用于他们。其他项目涉及向合作伙伴或供应商发送数据。这些需要额外的计划来安排使用他们的测试系统进行测试,并获得他们对数据是否有效和格式是否正确的反馈。

Lisa’s team has written several interfaces to allow users to upload data to their systems. In each case, they had to get the proposed file format out to the users early to make sure it would work for them. Other projects involved sending data to partners or vendors. These required extra planning to arrange testing with their test systems and getting their feedback on whether data was valid and correctly formatted.

如果您使用第三方产品作为解决方案的一部分,您可能会假设它已经过测试,但这可能是一个糟糕的假设。您将需要预算额外的时间来结合供应商软件测试您的应用程序。如果其他公司的软件有问题,可能需要很长时间才能解决。Lisa 的团队使用第三方软件来完成文档创建等关键任务。如果主题包括修改或创建新文档,他们会计划额外的时间在需要时升级软件,以及额外的需要时间进行测试以防需要修复。如果可能,尽早将第三方软件引入项目,并开始端到端测试。您可以使用该界面的次数越多,您的生活就会越好。

If you’re using a third-party product as part of your solution, you might assume it has been tested, but that might be a poor assumption. You will need to budget extra time to test your application in conjunction with the vendor software. If there’s a problem in the other company’s software, it might take a long time to resolve. Lisa’s team uses third-party software for critical tasks such as document creation. If a theme includes modifying or creating new documents, they plan extra time to upgrade the software if needed, and extra time for testing in case fixes are needed. If possible, bring the third-party software into the project early, and start end-to-end testing. The more you can work with the interface, the better off you’ll be.

我们经常忘记直到为时已晚的其他第三方软件是我们自己的测试环境。有时,一个团队会合并新代码,以利用他们所选语言的新功能。例如,如果团队成员使用 AJAX 或 JavaScript,他们可能需要升级他们正在使用的软件开发工具包。这意味着团队还必须升级其生产运行时环境,因此请考虑到这一点并尽早进行测试。

Other third-party software that we often forget about until it’s too late is our own testing environments. Sometimes a team will incorporate new code that takes advantages of new features in their chosen language. For example, if team members are using AJAX or JavaScript, they may need to upgrade the software development kit they’re using. This means that a team will have to upgrade its production runtime environment as well, so take that into consideration and test early.

客户或合作伙伴可能对不在您自己团队范围内的发布感到担忧。Lisa 的团队曾在最后一刻被阻止发布一项新功能,因为合作伙伴没有时间与他们的法律顾问就变更达成一致。程序员必须快速设计出一种方法来关闭该功能,而无需进行大量的额外测试。有趣的是,不使用敏捷开发的合作伙伴有时会在截止日期前遇到困难。当您的团队在截止日期前完成时,他们可能毫无准备。

Clients or partners might have concerns about a release that isn’t within your own team’s scope. Lisa’s team was once prevented at the very last minute from releasing a new feature because a partner didn’t have time to okay the change with their legal advisors. The programmers had to quickly devise a way to turn the functionality off without requiring extensive additional testing. Interestingly, partners who aren’t using agile development sometimes have trouble meeting their own deadlines. They might be unprepared when your team meets the deadline.

珍妮特的故事

Janet’s Story

我参与了一个项目来实现一项功能,该功能需要新的硬件来扫描新的二维条码。该团队决定分阶段实施,因为不知道扫描仪何时可用于全面测试,但客户希望在扫描仪到达时准备好代码。

I worked on a project to implement a feature that required a new piece of hardware for scanning a new 2D bar code. The team decided to implement in stages because it was not known when the scanners would be available for full testing, but the customer wanted the code ready for when the scanners arrived.

初始阶段是程序员密集型阶段,因为有很多研究要做。在他们确定了如何实现该功能后,创建了故事以将其添加到代码中。但是,我们知道在扫描仪可用之前我们无法对其进行彻底测试。代码已准备好进行测试,但我们无需全部支持,只需担心测试是否可以为发布关闭该功能。下一个版本将需要更多测试,但前提是扫描仪可用。故事的测试保存在产品待办事项列表中,因此我们不会忘记这样做。

The initial phase was programmer-intensive because there was a lot of research to be done. After they determined how they would implement the feature, the story was created to add it into the code. However, we knew we couldn’t thoroughly test it until the scanners were available. The code was ready to test, but instead of backing it all out, we only needed to worry about testing that the feature could be turned off for the release. The next release would require more testing, but only if the scanners were available. The testing of the story was kept in the product backlog so we would not forget to do it.

——珍妮特

—Janet

如果您将与其他团队合作开发同一系统或相关系统的不同组件,请预留时间与他们协调。从每个团队中指定一名成员一起协调是个好主意。

If you’ll be working with other teams developing different components of the same system, or related systems, budget time to coordinate with them. It’s a good idea to designate a member from each team to coordinate together.

发布计划是确定团队中需要的额外角色、额外资源以及异常情况所需时间的时间。

Release planning is the time to identify extra roles you need on your team, additional resources, and time needed for out-of-the-ordinary circumstances.

测试计划

Test Planning

我们不能期望在详细级别上计划发布中的迭代。我们可以了解主题的钢丝线,确定故事的优先级,并猜测哪些故事将出现在哪个迭代中。详细的测试计划需要等待迭代计划。尽管如此,我们仍需要考虑高层次的测试,并尝试为其预留足够的时间。我们甚至可能会在发布计划会议之外花时间来制定我们的发布测试策略。在第 8 章支持团队的面向业务的测试中,我们提到了敏捷测试的危险之一:“忘记大局”。测试计划将帮助您解决这个问题。

We can’t expect to plan the iterations in a release at a detailed level. We can get an idea of the theme’s steel threads, prioritize stories, and make a guess at what stories will be in which iteration. Detailed test planning needs to wait for iteration planning. Still, we need to think about testing at a high level, and try to budget enough time for it. We might even take time separately from the release planning meeting to strategize our testing for the release. In Chapter 8, Business-Facing Tests that Support the Team, we mentioned one of the perils of agile testing: “forgetting the big picture.” Test Planning will help you with that problem.

第 8 章,“支持团队的面向业务的测试”,解释了如何识别故事或主题中的钢线或薄片。

Chapter 8, “Business-Facing Tests that Support the Team,” explains how to identify steel threads or thin slices in a story or theme.

从哪儿开始

Where to Start

在发布计划期间,了解每个故事或高级用户验收测试用例的业务满意度条件很有帮助。当故事需要澄清时,敏捷测试人员会要求提供示例。在此阶段,示例将是高级示例,仅涵盖基础知识,但足以确定故事的大小和优先级。绘制流程图或在白板上写下计算结果并进行讨论有助于我们确定项目特定的测试问题。

During release planning, it’s helpful to know the business conditions of satisfaction for each story or high-level user acceptance test case. When stories need clarification, agile testers ask for examples. At this stage, examples will be high-level, covering just the basics, but enough to be able to size and prioritize the story. Drawing flowcharts or writing calculations on the whiteboard and discussing them helps us identify project-specific testing issues.

至少,团队需要了解计划首先执行的最高优先级故事。轻量级计划可能只涉及查看那些核心故事,并了解需要更多时间来定义额外的测试。

At a minimum, the team needs to understand the top-priority stories that are scheduled to be performed first. Lightweight planning might involve only looking at those core stories with the understanding that more time will be needed for defining additional tests.

当我们了解哪些故事可能会包含在发布中时,我们可以开始考虑测试的范围。做出了哪些可能影响测试的假设?使用第三方软件,例如使用航运公司的航运计算API的例子,影响测试计划。此版本中是否存在任何会影响测试的异常风险?如果我们有实现批处理作业的故事,并且我们以前从未在系统中进行过任何批处理,那么可能会有影响测试的新框架。我们需要预算时间来学习它们。

As we get a sense of which stories will probably be included in the release, we can start thinking about the scope of the testing. What assumptions have been made that might affect testing? Use of third-party software, such as the example of using a shipping company’s shipping calculation API, affects test planning. Are there any unusual risks in this release that will impact testing? If we have stories to implement batch jobs, and we’ve never had any batch processing in the system before, there are probably new frameworks that impact testing. We need to budget time to learn them.

为什么要编写测试计划?

Why Write a Test Plan?

在发布计划中,我们讨论发布的目的、范围内的内容以及我们所做的假设。我们进行一些快速的风险分析并计划我们的测试方法来解决这些风险。我们考虑自动化以及我们对测试环境和测试数据的需求。我们当然想确定里程碑和可交付成果。嗯,这听起来像是。. . 一个测试计划!

In release planning, we talk about the purpose of the release, what’s in scope, and what assumptions we’re making. We do some quick risk analysis and plan our test approach to address those risks. We consider automation and what we need for test environments and test data. We certainly want to identify milestones and deliverables. Hmmm, this is starting to sound like . . . a test plan!

如果像我们一样,您花时间在传统的瀑布环境中工作,您可能会浪费时间编写没有人阅读也没有人费心维护的庞大测试计划。在敏捷开发中,我们希望我们的测试计划能够满足我们的需求。出于合规性原因,您的客户可能需要每个版本的测试计划。即使它不是必需的可交付成果,它也很有用。保持简洁和轻便。它只需要在此版本中为您服务。解决特定于此版本或项目的测试问题。包括您的风险分析并确定假设。概述您的客户已确定的关键成功因素。考虑人们需要了解与测试相关的内容,并删除任何无关的内容。

If, like ourselves, you spent time working in a traditional waterfall environment, you might have wasted time writing bulky test plans that nobody read and nobody bothered to maintain. In agile development, we want our test plans to serve our needs. Your customer might require a test plan for each release for compliance reasons. Even if it’s not a required deliverable, it can be useful. Keep it concise and lightweight. It only has to serve your purposes during this release. Address the testing issues that are specific to this release or project. Include your risk analysis and identify assumptions. Outline the critical success factors that your customer has identified. Think about what people need to know related to testing, and remove anything extraneous.

有关测试计划和测试策略的更多信息,请参阅第 5 章转换典型过程”。

See Chapter 5, “Transitioning Typical Processes,” for more about test plans and test strategies.

即使您没有创建正式的测试计划,也请确保您已记下发布中涉及的所有这些不同的测试因素。在每次迭代计划会议期间,您都希望将它们牢记在心。测试计划的最大好处是计划本身。它允许您考虑和解决诸如测试数据要求、基础设施甚至需要什么测试结果等问题。测试计划是一种风险缓解策略。让我们考虑其中的一些问题。

Even if you don’t create a formal test plan, be sure you have made note of all these different testing factors involved in the release. You’ll want to keep them in mind during every iteration planning session. The biggest benefit in test planning is the planning itself. It allows you to consider and address issues such as test data requirements, infrastructure, or even what test results are required. Test planning is a risk mitigation strategy. Let’s consider some of these issues.

测试类型

Types of Testing

第 III 部分中,我们介绍了测试的四个象限,并讨论了您在项目中可以进行的所有不同类型的测试。发布计划是考虑这些不同需求的好时机。您是否需要计划引入负载测试工具,或者是否需要构建某种专业测试工具?

In Part III, we covered the four quadrants of testing and talked about all of the different types of testing you can do during your project. Release planning is a good time to consider these different needs. Do you need to plan to bring in a load test tool, or will there be the need to build some kind of specialty test harness?

可能你的下一个版本只是你上一个版本的扩展,你将继续创建你的例子,自动化你的故事测试,并像你一直在做的那样做其余的测试。你是幸运者之一。对于那些开始一个全新项目且之前没有任何流程的人来说,现在是时候考虑您需要进行哪些测试了。我们并不是说您必须决定如何测试每个故事,而是要着眼大局并考虑象限。您是否需要计划一个特殊的 UAT,或者迭代演示是否足够?尽早提出这些问题很重要,这样团队才能为它们做好计划。

It could be that your next release is just an extension of your last, and you will just carry on creating your examples, automating your story tests, and doing the rest of the testing as you’ve been doing. You are one of the lucky ones. For those of you who are starting a brand new project with no previous processes in place, now is the time to consider what testing you will need. We don’t mean you have to decide how to test each story, but look at the big picture and think about the quadrants. Will you need to plan for a special UAT, or will the iteration demos be enough? It is important to raise these issues early so the team can plan for them.

基础设施

Infrastructure

在进行测试计划时,您需要考虑您的基础设施。基础设施可能意味着您的持续集成设置、测试环境和测试数据库。这可能意味着您如何推广您的构建到您的测试环境。这可能意味着你的测试实验室,如果你有一个,或者有一个单独的服务器来运行你所有的自动化测试。这些通常是需要一些准备时间才能到位的基础设施。这是制定计划的时候了。

While you are doing your test planning, you need to consider your infrastructure. Infrastructure can mean your continuous integration setup, test environments, and test database. It can mean how you promote your builds to your test environments. It might mean your test lab, if you have one, or having a separate server to run all your automation tests. These are generally pieces of infrastructure that need some lead time to get in place. This is the time to make a plan.

丽莎的故事

Lisa’s Story

某些类型的测试可能需要额外的努力。我的团队有一个工具来进行性能测试和一些脚本,但我们缺乏一个生产风格的环境,我们可以控制所有可能影响性能的变量。例如,测试数据库由测试人员、程序员和两个构建过程共享。较慢的性能可能仅仅意味着有人正在运行数据库密集型测试。我们使用暂存环境来获得基线,但它缺少一些生产组件。我们设定了六个月的目标,为适当的测试环境购买硬件和软件并进行设置。我们在每次迭代中写了一两张任务卡来逐步建立环境。

Some types of testing might require extra effort. My team had a tool to do performance testing and some scripts, but we lacked a production-style environment where we could control all of the variables that might affect performance. For example, the test database was shared by testers, programmers, and two build processes. Slower performance might simply mean someone was running database-intensive tests. We used our staging environment to get a baseline, but it was missing some of the components of production. We set a six-month goal to acquire hardware and software for a proper test environment and get it set up. We wrote a task card or two each iteration to establish the environment step by step.

——丽莎

—Lisa

无论您的需求是什么,请确保您了解它们并且可以针对您的需求进行规划。如果您没有合适的基础架构,那么您将浪费时间尝试将其组合在一起,并在迭代中期造成瓶颈。

Whatever your needs are, make sure you understand them and can plan for what you need. If you don’t have the right infrastructure, then you will waste time trying to get it together and cause a bottleneck in mid-iteration.

测试环境

Test Environments

当我们查看下一个版本中的功能类型时,我们可能会发现需要一个全新的测试环境。考虑您可能还需要的专门测试环境。您需要更多工具吗?您是否需要扩展您的测试实验室以便您可以使用不同的浏览器和操作系统进行测试?现在是考虑所有测试注意事项的时候了。

As we look at the types of features in the next release, we might see the need for a whole new test environment. Think about specialized test environments you may need as well. Will you need more tools? Do you need to expand your test lab so that you can test with different browsers and operating systems? This is the time to think about all testing considerations.

如果您正在计划您的第一个版本,测试环境是一个关键的考虑因素。您可能需要一个故事或迭代来设置您需要的基础设施。我们已经启动了不止一个项目,我们唯一可以测试的地方就是开发环境。我们发现效果不是很好,因为环境永远不够稳定,无法进行有效测试。

If you’re planning your first release, test environments are a key consideration. You might need a story or iteration just to set up the infrastructure you need. We’ve started more than one project where the only place we could test was the development environment. We found that doesn’t work very well, because the environment is never stable enough for effective testing.

就像程序员有自己的沙箱来工作和测试一样,如果每个测试人员都具有相同的可用性和控制权,它就会运作良好。我们认识到并非所有应用程序都适用于此,但至少,您需要知道您正在测试的是什么版本。您还需要其他人不会绕过他们的测试的测试数据。如果您没有自己控制的测试沙箱,请花时间计划您需要为测试环境建立的内容。与您的团队集思广益,商讨如何获得必要的硬件和软件。这可能需要时间,因此制定一个 B 计划,在等待您需要的基础设施的同时完成一些事情。

Just as programmers have their own sandboxes to work and test in, it works well if each tester has that same availability and control. We recognize that not all applications lend themselves to this, but at the very least, you need to know what build you’re testing. You also need test data that others will not walk over with their tests. If you don’t have a testing sandbox that’s under your own control, take time to plan what you need to establish for your test environments. Brainstorm with your team about how you can obtain the necessary hardware and software. It might take time, so develop a Plan B for getting something done while waiting for the infrastructure you need.

如果您在大型系统上工作,您可能不得不与其他团队一起排队,以便在测试或暂存环境中获得时间,其中包括您必须使用的所有各种软件。此暂存环境应尽可能模仿您的生产系统。如果您的组织没有专人负责创建环境,您的团队可能需要额外的角色来专门获取您需要的测试环境。这些角色也可能涉及与其他团队合作。发布计划是时候考虑所有这些测试基础架构需求了。

If you’re working on a large system, you may have to queue up along with other teams to get time on a test or staging environment that includes all of the various pieces of software with which yours must work. This staging environment should mimic your production system as much as possible. If your organization doesn’t have someone responsible for creating environments, your team might need extra roles dedicated to obtaining the test environments you need. These roles might involve working with other teams as well. Release planning is the time to consider all of these testing infrastructure requirements.

测试数据

Test Data

发布或主题规划也是考虑项目期间可能需要哪些测试数据的好时机。

Release or theme planning is also a good time to think about what test data you might need during the project.

使用与真实数据非常相似的测试数据通常是一种很好的做法。规划您需要的数据。我们有机会在多个组织中使用生产数据的副本。真实数据为探索性测试的不同场景提供了良好的基础。生产数据在用于测试之前可能需要“清理”,以删除任何敏感信息,例如身份或银行帐号。需要更改数据以隐藏原始值但保持有效,以免违反数据库限制。由于数据库专家将生产数据移植到测试环境需要时间,因此请确保将它们包含在您的计划中。

Using test data that closely resembles real data is generally a good practice. Plan for the data you need. We’ve had the opportunity in several organizations to use a copy of production data. Real data provides a good base for different scenarios for exploratory testing. Production data may need to be “scrubbed” before it’s used for testing in order to remove any sensitive information such as identification or bank account numbers. The data needs to be altered to hide the original values but remain valid so that it doesn’t violate database restrictions. Because it takes time for database experts to port production data to a test environment, be sure they’re included in your planning.

珍妮特的故事

Janet’s Story

在与我合作的一个组织中,我们使用了两种不同的基线测试数据方案。对于我们单独的测试环境,我们使用 Fit fixtures 来加载预定义的数据。我们试图让这些数据尽可能接近生产,但我们也为它植入了一些非常具体的测试数据。每次我们签出新版本的代码时,我们都能够重新加载一组基本数据。通过这种方式,我们还测试了数据库模式,看看是否有任何变化。

In one of the organizations I was working with, we used two different baseline test data schemes. For our individual test environments, we used Fit fixtures to load predefined data. We tried to make this data as close to production as possible, but we also seeded it with some very specific test data. Every time we checked out a new version of code, we were able to reload a base set of data. In this way, we also tested the database schema as well to see if anything had changed.

为了我们希望数据持久化的更稳定的测试环境,我们使用了程序员在更改数据库时开发的数据迁移脚本。这些迁移脚本最终被用于最初的生产切换,到那时我们非常确定它们是正确的。

For our more stable test environment where we wanted data persisted, we used the data migration scripts that the programmers developed as they made database changes. These migration scripts were eventually used for the initial cut over from production and by then we were pretty certain they were correct.

——珍妮特

—Janet

争取客户的支持以获取有意义的测试数据。如果您正在处理涉及将文件发送给第三方供应商的故事,您的业务专家可以找出供应商希望文件中包含哪些数据。Lisa 的团队开发了允许退休计划经纪人向其客户提供共同基金组合的功能。他们要求产品负责人提供投资组合样本,包括每个投资组合的名称、描述和一组基金。这有助于他们使用真实数据进行测试。

Enlist your customers’ support in obtaining meaningful test data. If you’re working on a story that involves sending a file to a third-party vendor, your business expert can find out what data the vendor expects in the file. Lisa’s team developed features to allow retirement plan brokers to offer their customers portfolios of mutual funds. They asked the product owner to provide samples of portfolios, including a name, description, and set of funds for each. This helped them test with realistic data.

随着时间的推移,测试数据往往会变得陈旧和过时。较旧的数据,即使来自生产,也可能不再准确反映当前的生产数据。使用不再有效的数据进行的“通过”测试会给人一种误导性的自信感。持续审查您的测试数据需求。根据需要使用新方法刷新数据或创建数据。

Test data tends to get stale and out of date over time. Older data, even if it came from production, may no longer accurately reflect current production data. A “passing” test using data that’s no longer valid gives a misleading sense of confidence. Continually review your test data needs. Refresh data or create it using a new approach, as needed.

测试数据要求因测试类型而异。回归测试通常可以创建自己的数据或针对可以快速刷新到已知状态的小型代表性数据集运行。探索性测试可能需要生产类型数据的完整副本。

Test data requirements vary according to the type of testing. Regression tests can usually create their own data or run against a small representational set of data that can be refreshed to a known state quickly. Exploratory testing may need a complete replica of production type data.

第 14 章,“敏捷测试自动化策略”,探讨了获取测试数据的不同方法

Chapter 14, “An Agile Test Automation Strategy,” explores different approaches to obtaining test data

检测结果

Test Results

不同的团队对测试结果报告有不同的要求。考虑一下您将如何在游戏的这个阶段报告测试结果,以便在进行实际报告时可以有效地这样做。您的组织可能有审计合规性要求,或者您的客户可能只是想知道您是如何测试的。了解您的需求,以便您可以选择适合您团队的方法。

Different teams have different requirements for test result reporting. Think about how you are going to report test results at this stage of the game so that you can do so effectively when the time comes to do the actual reporting. Your organization may have audit compliance requirements, or maybe your customer just wants to know how you tested. Understand your needs so that you can choose the approach that is right for your team.

报告测试结果的方式有很多种。有一些供应商工具可以记录自动和手动结果。您的团队可能会找到一种方法来保存来自 Fit 等工具的结果,或者您可能只是选择保留一个大的可见的手动图表。

There are many ways to report test results. There are vendor tools that will record both automated and manual results. Your team may find a way to persist the results from tools such as Fit, or you may just choose to keep a big visible manual chart.

一些团队采用的方法是创建本地测试结果应用程序。例如,用Ruby on Rails 为数据库编写的一个简单的Ruby 应用程序或一个带有PHP 前端的MySQL 数据库都可以构成一个非常简单但易于使用的测试管理系统。

The approach that a few teams have taken is to create home-grown test result applications. For example, a simple Ruby application written with Ruby on Rails for the database or a MySQL database with a PHP front end can make a very simple but easy-to-use test management system.

诸如此类的工具可以非常简单,也可以包含额外的复杂性,例如对测试进行分类的功能。重要的是测试结果。如果您的自动化测试记录了它们通过或失败的结果以及错误,您就有一些历史可以帮助确定测试的脆弱性。

A tool such as this can be very simple or can include added complexity such as the capability to categorize your tests. The important thing is the test results. If your automated tests record their pass or fail result along with the error, you have some history to help determine fragility of the test.

您的团队可以配置您的自动构建过程,以通过电子邮件或团队成员可以在线查看的反馈实用程序或 Web 界面提供每个构建的测试结果。随着时间的推移,结果可以用多种格式进行总结,使进展可见。Lisa 的一个团队制作了一份每日图表,记录了编写、运行和通过的测试,并将其张贴在团队的工作区中。另一个生成了一个日历,其中包含每天通过的单元测试的数量。即使是简单的视觉效果也是有效的。

Your team can configure your automated build process to provide test results from each build, by email, or a feedback utility or web interface that team members can view online. Results over time can be summarized in a variety of formats that make progress visible. One of Lisa’s teams produced a daily graph of tests written, run, and passing that was posted in the team’s work area. Another produced a daily calendar with the number of unit tests passing every day. Even simple visual results are effective.

我们将在本章后面讨论一些您可以使用的指标。

We talk about some of the metrics you can use later in this chapter.

测试计划备选方案

Test Plan Alternatives

我们已经讨论了为什么要测试计划以及您应该考虑什么。现在我们讨论一些您可能习惯的繁重测试计划的替代方案。无论您的组织使用何种类型的测试计划,都将其变成您的。以有利于您的团队的方式使用它,并确保您满足客户的需求。与您的团队制作的任何文档一样,它应该实现一个目的。

We’ve talked about why to test plan and what you should consider. Now we talk about some of the alternatives to the heavy test plans you may be used to. Whatever type of test plan your organization uses, make it yours. Use it in a way that benefits your team, and make sure you meet your customer’s needs. As with any document your team produces, it should fulfill a purpose.

轻量级测试计划

Lightweight Test Plans

如果您的组织或客户坚持针对 SOX 合规性或其他法规需求的测试计划,请考虑一个涵盖必需品但不包含任何额外内容的轻量级测试计划。不要重复已经包含在项目计划或项目章程中的项目。示例测试计划可能看起来像图 15-6中所示的那样。

If your organization or customer insists on a test plan for SOX compliance or other regulatory needs, consider a lightweight test plan that covers the necessities but not any extras. Do not repeat items that have already been included in the Project Plan or Project Charter. A sample Test Plan might look something like the one shown in Figure 15-6.

图 15-6 示例测试计划

Figure 15-6 Sample Test Plan

图像

测试计划不应涵盖所有可能发生的情况或每个故事,也无意解决可追溯性问题。它应该是一个帮助您考虑测试项目风险的工具。它不应取代与客户或团队其他成员的面对面交谈。

A test plan should not cover every eventuality or every story, and it is not meant to address traceability. It should be a tool to help you think about testing risks to your project. It should not replace face-to-face conversation with your customer or the rest of your team.

使用测试矩阵

Using a Test Matrix

Janet 使用发布计划与测试人员和客户一起开发高级测试矩阵。测试矩阵是一种简单的方式来传达有关您要测试的功能的大局。它可以让您的团队快速了解所需的测试。

Janet uses release planning to work with the testers and customers to develop a high-level test matrix. A test matrix is a simple way to communicate the big picture concerning what functionality you want to test. It gives your team a quick overview of the testing required.

测试矩阵只是侧面的功能列表和顶部的测试条件。在考虑测试条件和功能时,请考虑整个应用程序以及新功能或更改功能的任何影响可能对应用程序的其余部分有影响。测试人员与客户坐在一起思考测试条件是很重要的。

A test matrix is just a list of functionality down the side and test conditions across the top. When thinking about test conditions and functionality, consider the whole application and any impact the new or changed functionality might have on the rest of the application. Testers sitting with customers and thinking about test conditions is what is important.

它也可以是一种跟踪覆盖率的机制,并且可以尽可能详细。团队可以使用高级测试矩阵向客户团队或管理层展示已经测试过的内容和剩余的内容。团队可以使用更详细的测试矩阵来显示测试计划和跟踪测试进度。创建矩阵后,在完成测试时填充方块变得很容易。把事情简单化。因为我们喜欢易于阅读的大图表,所以我们推荐对您的团队有意义的颜色。例如,绿色 (G) 表示测试已完成并且团队对此感到满意,而黄色 (Y) 可能表示已经完成了一些测试,但如果有时间还需要进行更多的探索性测试。红色 (R) 表示有东西坏了。白色方块表示尚未测试,

It can also be a mechanism to track coverage and can be as detailed as you like. A high-level test matrix can be used by the team to show the customer team or management what has been tested already and what is left. A more detailed test matrix can be used by the team to show what is planned for testing and track the progress of the testing. After the matrix has been created, it becomes easy to fill in the squares when testing is done. Keep it simple. Because we like big visible charts that are easy to read, we recommend colors that mean something to your team. For example, green (G) means testing is done and the team is happy with it, while yellow (Y) might mean some testing has been done but more exploratory testing is needed if there is time. Red (R) means something is broken. A white square means it hasn’t been tested yet, and a gray (not applicable) square means it doesn’t need to be tested.

让我们看一个例子。我们有一个小版本要发布,用于计算运费。在图 15-7中,不同的功能部分在一个轴上表示,而装运的属性在另一个轴上表示。单个单元格采用颜色编码,以显示哪些案例已经过测试,哪些需要更多关注。“<= 2 lbs”的所有单元格都已完成,> 4 lbs 的前三个单元格已完成但需要更多探索性测试,“Ship to Alaska”/“>4 lbs”单元格表示可能存在问题。

Let’s look at an example. We have a small release we want to put out that calculates shipping costs. In Figure 15-7, different pieces of functionality are represented on one axis, and properties of the shipment are represented on the other. Individual cells are color-coded to show which cases are tested and which need more attention. All of the cells for “<= 2 lbs” are finished, the top three cells for > 4 lbs are done but need more exploratory testing, and the “Ship to Alaska”/“>4 lbs” cell denotes a possible issue.

图 15-7样本测试矩阵

Figure 15-7 A sample test matrix

图像

珍妮特的故事

Janet’s Story

我在一个项目中使用测试矩阵产生了意想不到的副作用。客户和测试人员将测试矩阵放在一起,并考虑了项目的所有受影响的功能以及他们需要的高级测试条件。正如预期的那样,计划行为带来了很多问题,这些问题直到后来才被忽略。

I had an unexpected side effect from using a test matrix in one project I was on. The customers and testers put the test matrix together, and had thought of all affected functionality for the project and the high-level test conditions they would need. As expected, the act of planning brought a lot of issues out that would have been missed until later.

当他们将矩阵挂在他们团队区域的墙上时,开发团队负责人 Dave 表达了兴趣。其中一位测试人员向他解释了矩阵,当他说这对他们也非常有用时,我感到很惊讶。Dave 说:“我不知道这个功能会影响这个区域。我们需要确保我们的单元测试也涉及到这一点。”

When they hung the matrix on the wall in their team area, Dave, the developer team lead, expressed an interest. One of the testers explained the matrix to him, and I was surprised when he said it was very useful for them as well. Dave said “I didn’t know that this functionality would affect this area. We need to make sure our unit tests touch on this as well.”

回想起来,我本不应该感到惊讶,但我以前从未有过与程序员的那种经历。

Looking back on this, I shouldn’t have been surprised, but I had never had that experience with the programmers before.

——珍妮特

—Janet

测试矩阵是一个非常强大的工具,如果您的团队有这些问题,可用于帮助解决可追溯性问题。考虑什么对您的团队有意义,并根据您的团队和您自己的情况进行调整。

A test matrix is a very powerful tool and can be used to help address traceability issues if your team has those problems. Think about what makes sense for your team and adapt it for your team and what makes sense to you.

测试电子表格

Test Spreadsheet

珍妮特还看到了电子表格格式的使用并取得了一些成功。例如,在 WestJet,工作簿中的第一个选项卡是应用程序中存在的高级功能列表。对于每一行,团队确定该项目是否影响了该功能。如果是这样,他们给出了预期影响的评级。在确定更改的影响之后,就可以做出有关测试环境、测试数据或 UAT 的决策。

Janet has also seen a spreadsheet format used with some success. For example, at WestJet, the first tab in a workbook was a high-level list of functionality that existed in the application. For each row, the team determined if the project affected that piece of functionality. If so, they gave a rating of the expected impact. After the impact of the changes had been determined, decisions about test environments, test data, or UAT could then be made.

选项卡用于风险和假设,但可用于您的团队可能需要的任何内容。电子表格等灵活的格式意味着您可以对其进行定制以适合您。

Tabs were used for risks and assumptions but could be used for anything your team may need. A flexible format such as a spreadsheet means you can tailor it to work for you.

该信息可以多种不同方式使用。它可用于确定将探索性测试工作集中在何处,或者可能有助于创建高级测试矩阵以确保您在测试期间接触到所有区域。

This information can be used in a number of different ways. It can be used to determine where to concentrate your exploratory testing efforts, or maybe to help create a high-level test matrix to make sure you touch on all of the areas during your testing.

一块白板

A Whiteboard

如果您的团队是非正式的并且发布的版本很少,那么任何类型的文档都可能太多了。有时在白板或索引卡上列出风险和假设就足够了。珍妮特使用白板来管理风险,它运作良好。如果风险真的成为一个问题,结果会被记录下来并划掉。添加新的风险和缓解策略很容易,而且整个团队都可以看到该列表。这也可以在 wiki 页面上完成。

If your team is informal and has small releases, any kind of documentation may be too much. Sometimes it’s enough to list the risks and assumptions on a whiteboard or on index cards. Janet has used a whiteboard to manage risks, and it worked quite well. If a risk actually became an issue, the result was documented and crossed off. It was easy to add new risks and mitigation strategies, and the list was visible to the whole team. This could also be done on a wiki page.

我们怎么强调您都需要了解您的团队及其需求。

We cannot stress enough that you need to know your team and its needs.

自动化测试列表

Automated Test List

有时您可能需要向客户提供更多信息,例如测试用例列表。如果您的团队有一个工具,您可以从中提取测试用例名称列表,您可以轻松地将此列表提供给需要它的任何人。这将提供更多传统类型的详细测试计划,但只有在实际编写测试后才能使用。我们不建议在这上面花任何时间,因为我们看不到附加值,但有时可能需要此列表进行风险评估或审计。

Sometimes you may be required to present more information to your customers, such as a list of test cases. If your team has a tool from which you could extract a list of test case names, you could provide this list easily to anyone who needed it. This would present more of a traditional type detailed test plan but wouldn’t be available until after tests were actually written. We don’t recommend spending any time on this because we don’t see added value, but sometimes this list may be required for risk assessment or auditability.

为可见性做准备

Preparing for Visibility

如果您的团队刚刚开始敏捷开发,请确保为早期迭代准备好必要的基础设施。您可以在进行过程中改变跟踪进度的方式,而您的回顾将帮助您发现这些问题。如果您在完成为每次迭代计划的工作时遇到问题,也许您需要更多可见的图表或视觉辅助工具来帮助您衡量进度并进行迭代中期调整。您的客户是否有办法了解迭代的进展情况以及完成了哪些故事?在每次迭代之前花点时间评估您是否获得了正确的反馈以跟踪测试。

If your team is just getting started with agile development, make sure you have necessary infrastructure in place for your early iterations. You may change the way you are tracking progress as you go along, and your retrospectives will help you bring these issues to light. If you’re having problems completing the work planned for each iteration, maybe you need more visible charts or visual aids to help you gauge progress and make mid-iteration adjustments. Do your customers have some way to know how the iteration is progressing and which stories are done? Take time before the each iteration to evaluate whether you’re getting the right kind of feedback to keep track of testing.

跟踪测试任务和状态

Tracking Test Tasks and Status

我们所知道的高效敏捷团队都遵循这个简单的规则:“故事只有经过测试才能完成。” 这条规则可以扩展为不仅必须测试故事,还必须签入代码,它必须具有由持续构建过程运行的自动化测试,它必须被记录下来,或者任何你团队的“完成”标准是。在迭代期间的任何时候,您都需要能够快速评估每个故事还剩下多少测试工作,以及哪些故事已经“完成”。故事板或任务板是实现此目的的理想选择,尤其是当它们使用颜色编码来表示测试任务与开发和其他类型的任务时。软木板、带磁铁的钢板、海报大小的便利贴或白板都可以。给每个故事单独一行,并按优先级排序。有“待办事项”、“进行中的工作”、“验证、

The effective agile teams we know all follow this simple rule: “No story is done until it’s tested.” This rule can be expanded to say that not only must the story be tested, the code must be checked in, it must have automated tests that are run by a continual build process, it must be documented, or whatever your team’s “doneness” criteria are. At any time during an iteration, you need to be able to quickly assess how much testing work remains on each story, and which stories are “done.” Story or task boards are ideal for this purpose, especially if they use color-coding to denote test tasks vs. development and other types of tasks. Cork boards, steel sheets with magnets, poster-sized sticky notes, or whiteboards all work fine. Give each story its own row, and order them by priority. Have columns for “to do,” “work in progress,” “verify,” and “done.”

珍妮特的故事

Janet’s Story

我从团队成员开始,他们已经在只有几个程序员和一个测试人员的情况下进行了几个月的敏捷开发。他们一直在使用 XPlanner 来跟踪他们的任务和故事,并且对他们来说工作正常。在我上任的同时,增加了几个新程序员,站立会议变得不那么有效了;该团队没有完成其计划的故事。我建议使用故事板,尽管他们对保留两组“任务”持怀疑态度,但他们表示会尝试。

I started with team members who had been doing agile for a few months with only a couple of programmers and one tester. They had been using XPlanner to track their tasks and stories, and it was working ok for them. At the same time I came on board, a couple of new programmers were added, and the stand-ups became less effective; the team was not completing the stories it had planned. I suggested a storyboard, and although they were skeptical about keeping two sets of “tasks,” they said they would try it.

我们拿了一堵敞开的墙,用便利贴来制作我们的故事板。我们开始在故事板前站起来,我们的讨论变得更加具体。它提供了一种很好的可视化方式来了解任务何时完成以及还剩下什么要做。几个月后,团队再次壮大,我们不得不将故事板搬进办公室。我们还将站会和测试结果图表移到那里。然而,持续的可见性丢失了,程序员和测试人员停止移动他们的任务。

We took an open wall and used stickies to create our story board. We started having stand-ups in front of the story board and our discussion became more specific. It provided a nice visible way of knowing when the tasks were done and what was left to do. After a couple of months, the team grew again and we had to move the story board into an office. We also moved our stand-ups and our test result charts there. However, the constant visibility was lost, and programmers and testers stopped moving their tasks.

我们不得不重新评估我们想做什么。一种尺寸并不适合所有团队。确保您计划适合您的团队的内容。

We had to reevaluate what we wanted to do. One size does not fit all teams. Make sure you plan for what is right for your team.

——珍妮特

—Janet

一些团队针对不同类型的任务使用不同颜色的索引卡:绿色用于测试,白色用于编码,黄色和红色用于错误。其他团队每个开发任务使用一张卡片,并添加不同颜色的贴纸以表明测试正在进行或表明有错误需要解决。使用任何方法,让您快速了解有多少故事已“完成”,所有编码、数据库、测试和其他任务都已完成。随着迭代的进行,很容易看出团队是否走上正轨,或者您是否需要提出一个故事或让程序员参与测试任务。

Some teams use different colored index cards for the different types of tasks: green for testing, white for coding, yellow and red for bugs. Other teams use one card per development task, and add different colored stickers to show that testing is in progress or show that there are bugs to resolve. Use any method that lets you see at a quick glance how many stories are “done,” with all coding, database, testing, and other tasks completed. As the iteration progresses, it’s easy to see if the team is on track, or if you need to pull a story out or have programmers pitch in on testing tasks.

珍妮特的故事

Janet’s Story

我们的故事板(如图 15-8所示)不是很大,我们也没有太多的墙面空间来扩展常规的柱式任务板。相反,我们决定使用贴纸来指定状态。

Our story board (shown in Figure 15-8) wasn’t very big, and we didn’t have a lot of wall space to expand to have the regular column-type task board. Instead, we decided to use stickers to designate the status.

图 15-8故事板示例

Figure 15-8 Example story board

图像

白色卡片,如图15-8第一行所示,是常规任务,蓝色卡片指定技术故事,如重构或尖峰,而粉色卡片,显示在板的右侧,颜色最深,是需要解决的错误。很容易看出这张图片是在迭代开始时拍摄的,因为每张卡片上都没有彩色圆圈。在右上角,您可以看到图例。蓝色贴纸表示已编码,绿色表示已完成(已测试),红色表示任务未完成或错误被拒绝为未修复。当一个任务或故事完成时(即绿色贴纸),它被移动到板的右侧。

White cards, such as those shown in the first row of Figure 15-8, were regular tasks, blue cards designated technical stories such as refactoring or spikes, and pink cards, shown toward the right-hand side of the board as the darkest color, were bugs that need to be addressed. It is easy to see that this picture was taken at the beginning of an iteration because there are no colored circles on each card. In the top right-hand corner, you can see the legend. Blue stickers meant it has been coded, green would indicate done (tested), and red meant the task has been deemed not completed or a bug was rejected as not fixed. As a task or story was completed (i.e., green sticker), it was moved to the right of the board.

——珍妮特

—Janet

丽莎的故事

Lisa’s Story

四年多来,我们的故事板是几张金属板,涂上公司颜色,使用用磁铁贴在板上的颜色编码索引卡。图 15-9显示了迭代早期的图片。我们的任务卡也用颜色编码:开发任务为白色,编码任务为绿色,错误为黄色和红色,迭代中最初未计划的卡片为条纹。看板非常有效地表明了我们的进展,以至于我们最终不再为任务燃尽图而烦恼。它让我们专注于一次完成一个故事。我们还用它来发布其他大的可见图表,例如显示构建失败的大红色标志。我们喜欢我们的董事会。

For more than four years, our story board was a couple of sheets of sheet metal, painted in company colors, using color-coded index cards attached to the board with magnets. Figure 15-9 shows a picture of it early in an iteration. Our task cards were also color-coded: white for development tasks, green for coding tasks, yellow and red for bugs, and striped for cards not originally planned in the iteration. The board was so effective in indicating our progress that we eventually stopped bothering with a task burndown chart. It let us focus on completing one story at a time. We also used it to post other big visible charts, such as a big red sign showing the build had failed. We loved our board.

图 15-9另一个示例故事板。经 Mike Thomas 许可使用。版权所有 2008。

Figure 15-9 Another sample story board. Used with permission of Mike Thomas. Copyright 2008.

图像

然后,我们的一名团队成员搬到了海外。我们尝试将电子表格与物理故事板一起使用,但我们的远程队友发现电子表格太难使用了。我们尝试了几个为 Scrum 团队设计的软件包,但它们与我们真正的故事板太不一样了,我们无法适应使用它们。我们终于找到了一个产品(Mingle),它看起来和工作起来都非常像我们的物理板,每个人,包括我们的远程人员,都可以使用它。我们将旧的故事板涂成白色,现在我们可以在站立会议期间将故事板投影到墙上。

Then, one of our team members moved overseas. We tried using a spreadsheet along with our physical story board, but our remote teammate found the spreadsheet too hard to use. We tried several software packages designed for Scrum teams, but they were so different from our real story board that we couldn’t adjust to using them. We finally found a product (Mingle) that looked and worked enough like our physical board that everyone, including our remote person, could use it. We painted our old story board white, and now we can project the story board on the wall during stand-up meetings.

——丽莎

—Lisa

分布式团队需要某种在线故事板。这可能是一个电子表格,或者像 Mingle 那样模仿物理故事板的专用软件。

Distributed teams need some kind of online story board. This might be a spreadsheet, or specialized software that mimics a physical story board as Mingle does.

交流测试结果

Communicating Test Results

早些时候,我们谈到了计划如何跟踪测试结果。现在我们想谈谈如何有效地与他们沟通。测试结果是衡量进度的最重要方法之一,查看是否正在为每个故事编写和运行新测试,以及它们是否全部通过。一些团队会发布大的可见图表,显示已编写、运行和通过的测试数量。其他人让他们的构建过程通过电子邮件将自动测试结果发送给团队成员和利益相关者。一些持续集成工具提供 GUI 工具来监控构建和构建结果。

Earlier, we talked about planning how to track test results. Now we want to talk about effectively communicating them. Test results are one of the most important ways to measure progress, see whether new tests are being written and run for each story, and whether they’re all passing. Some teams post big visible charts of the number of tests written, run, and passed. Others have their build process email automated test results to team members and stakeholders. Some continuous integration tools provide GUI tools to monitor builds and build results.

我们听说有团队将投影仪连接到机器上,在连续构建中运行 FitNesse 测试并始终显示测试结果。测试结果是团队进步的具体描述。如果数量测试不会每天或每次迭代都进行,这可能表明存在问题。要么团队没有编写测试(假设他们正在开发测试优先),要么他们没有完成很多代码。当然,他们可能会删除旧代码和随之而来的测试。重要的是要分析趋势为何走向错误的方向。下一节将提供一些关于您可能希望收集和显示的指标类型的想法。

We’ve heard of teams that have a projector hooked up to the machine that runs FitNesse tests on a continuous build and displays the test results at all times. Test results are a concrete depiction of the team’s progress. If the number of tests doesn’t go up every day or every iteration, that might indicate a problem. Either the team isn’t writing tests (assuming they’re developing test-first), or they aren’t getting much code completed. Of course, it’s possible they are ripping out old code and the tests that went with it. It’s important to analyze why trends are going the wrong way. The next section gives you some ideas about the types of metrics you may want to gather and display.

无论您的团队决定他们想要传达您的进度,请确保您事先考虑好,并且每个人都能从中获益。

However your team decides they want to communicate your progress, make sure you think about it up front and everyone gets value from it.

发布指标

Release Metrics

我们在此处包含此部分,因为了解您希望从发布一开始就收集哪些指标非常重要。这些指标应该为您提供有关开发进展情况的持续反馈,以便您可以响应意外事件并根据需要更改您的流程。请记住,您需要了解您试图用您的指标解决什么问题,以便您可以跟踪正确的问题。我们在这里讨论的指标只是您可以选择跟踪的一些示例。

We include this section here, because it is important to understand what metrics you want to gather from the very beginning of a release. These metrics should give you continual feedback about how development is proceeding, so that you can respond to unexpected events and change your process as needed. Remember, you need to understand what problem you are trying to solve with your metrics so that you can track the right ones. The metrics we talk about here are just some examples that you may choose to track.

通过测试的次数

许多敏捷团队跟踪每个级别的测试数量:单元、功能、故事测试、GUI、负载等。趋势比数字更重要。看到测试数量增加,我们会有一种温暖而模糊的感觉。不过,没有上下文的数字只是一个数字。例如,如果一个团队说它有 1000 个测试,那是什么意思?1000 次测试的覆盖率是 10% 还是 90%?删除具有测试的代码时会发生什么?

Many agile teams track the number of tests at each level: unit, functional, story tests, GUI, load, and so on. The trend is more important than the number. We get a warm fuzzy feeling seeing the number of tests go up. A number without context is just a number, though. For example, if a team says it has 1000 tests, what does that mean? Do 1000 tests give 10% or 90% coverage? What happens when code that has tests is removed?

跟踪在故事级别编写、运行和通过的测试的数量是显示故事状态的一种方式。编写的测试数量显示了驱动开发的测试进度。了解有多少测试未通过,可以让您了解还需要编写多少代码。

Tracking the number of tests written, running, and passing at a story level is one way to show a story’s status. The number of tests written shows progress of tests to drive development. Knowing how many tests aren’t passing yet gives you an idea of how much code still needs to be written.

测试通过后,只要代码中存在功能,它就需要保持“绿色”。通过和失败的测试数量随时间变化的图表显示了回归失败是否存在问题,还显示了代码库的增长。同样,重要的是趋势。注意异常。

After a test passes, it needs to stay “green” as long as the functionality is present in the code. Graphs of the number of tests passing and failing over time show whether there’s a problem with regression failures and also show the growth of the code base. Again, it’s the trend that’s important. Watch for anomalies.

这些类型的测量可以简单地报告并且仍然有效。

These types of measurements can be reported simply and still be effective.

丽莎的故事

Lisa’s Story

我的团队每天通过电子邮件发送一个颜色编码的日历,显示带有全套回归测试的“完整构建”是否每天都通过(见图15-10)。连续两天“红色”(最深的颜色)引起管理层和开发团队的关注和注意。查看可视化测试结果有助于组织齐心协力修复失败的测试或导致构建无法运行的任何其他问题,例如硬件或数据库问题。

My team emails a color-coded calendar out every day showing whether the “full build” with the full suite of regression tests passed each day (see Figure 15-10). Two “red” days in a row (the darkest color) are a cause for concern and noticed by management as well as the development team. Seeing the visual test results helps the organization pull together to fix the failing tests or any other problems causing the build to not run, such as hardware or database issues.

——丽莎

—Lisa

图 15-10来自 Lisa 团队的完整构建结果电子邮件

Figure 15-10 Full build result email from Lisa’s team

图像

有不同的方法来衡量测试的数量。选择一个并尝试在所有类型的测试中保持全面一致,否则您的指标可能会变得混乱。衡量测试脚本或类的数量是一种方法,但每个都可能包含多个单独的测试用例或“断言”,因此计算这些可能更准确。

There are different ways to measure the number of tests. Choose one and try to stay consistent across the board with all types of tests, otherwise your metrics may get confusing. Measuring the number of test scripts or classes is one way, but each one may contain multiple individual test cases or “asserts,” so it may be more accurate to count those.

如果您要对测试进行计数,请务必报告该信息以便可以使用。构建电子邮件或构建状态 UI 可以传达不同级别的测试运行、通过和失败的数量。客户团队可能会满意仅在每个冲刺结束时、在冲刺回顾中或电子邮件中查看此信息。

If you’re going to count tests, be sure to report the information so that it can be used. Build emails or build status UIs can communicate the number of tests run, passed, and failed at various levels. The customer team may be content to see this information only at the end of each sprint, in the sprint review, or an email.

无论您选择收集什么指标,请确保团队接受它们。

Whatever metrics you choose to gather, be sure the team buys into them.

珍妮特的故事

Janet’s Story

我与一个已经进行了几年敏捷开发的团队签订了一份新合同,他们开发了大量的自动化功能测试。我开始记录每天通过的测试数量。当趋势显示通过的测试越来越少时,团队没有发现问题。单元测试得到维护并且正在做他们应该做的事情,因此团队对发布充满信心。似乎每次发布都会发生这种情况,团队会在发布前的最后一周花时间让所有测试都通过。维护测试的成本很高,但团队不想放慢速度来修复它们。除了我之外,每个人都对此表示满意。

I started a new contract with a team that had been doing agile for a couple of years, and they had developed a large number of automated functional tests. I started keeping track of the number of tests passing each day. The team didn’t see a problem when the trending showed fewer and fewer tests were passing. The unit tests were maintained and were doing what they were supposed to do, so the team felt confident in the release. It seemed this happened with every release, and the team would spend the last week before the release to make all of the tests pass. It was costly to maintain the tests, but the team didn’t want to slow down to fix them. Everyone was okay with this except me.

我没有看到在那个迟到的日期修复测试如何能够确保捕获正确的预期结果。我觉得我们冒着误报的风险。

I did not see how fixing the tests at that late date could ensure the right expected results were captured. I felt that we ran the risk of getting false positives.

在下一个发布周期开始时,我让团队同意在测试失败时尝试修复它们。没过多久,团队就意识到一旦我们知道测试被破坏就修复测试并不是那么困难,而且我们很早就发现了很多问题,这些问题通常要到很晚才被发现。团队很快设定了始终通过 95% 测试的目标。

At the start of the next release cycle, I got the team to agree to try fixing the tests as they broke. It didn’t take long for the team to realize that it wasn’t so tough to fix the tests as soon as we knew they were broken, and we found a lot of issues early that hadn’t usually been caught until much later. The team soon set a goal of having 95% of the tests passing at all times.

我们还意识到测试的脆弱性。该团队齐心协力重构了一些更复杂的测试并消除了多余的测试。随着时间的推移,高级测试的数量减少了,但质量和覆盖率却提高了。

We also realized how brittle the tests were. The team made a concerted effort to refactor some of the more complex tests and eliminate redundant ones. Over time, the number of high-level tests was reduced, but the quality and coverage was increased.

我们开始衡量通过率,但最终得到的要多得多。

We started out measuring passing rates, but we ended up with far more.

——珍妮特

—Janet

不要过于专注于实际测量结果,以至于您没有意识到趋势的其他副作用。如果需要,请对调整您正在测量的内容持开放态度。

Don’t get so caught up in the actual measurements that you don’t recognize other side effects of the trending. Be open to adjusting what you are measuring if the need is there.

代码覆盖率

代码覆盖率是另一个传统指标。我们的测试执行了多少代码?有优秀的商业和开源代码覆盖率工具可用,这些工​​具可以集成到您的构建过程中,以便您立即知道覆盖率是上升还是下降。与大多数指标一样,趋势是值得关注的。图 15-11显示了示例代码覆盖率报告。

Code coverage is another traditional metric. How much of our code is exercised by our tests? There are excellent commercial and open source code coverage tools available, and these can be integrated into your build process so that you know right away if coverage has gone up or down. As with most metrics, the trend is the thing to watch. Figure 15-11 shows a sample code coverage report.

表 15-11。来自 Lisa 团队的示例代码覆盖率报告。“Ghidrah”是新架构;“惠特尼”是遗留系统。

Table 15-11. Sample code coverage report from Lisa’s team. “Ghidrah” is the new architecture; “Whitney” is the legacy system.

图像

图 15-1215-13是共同作用的两个趋势示例。图 15-12显示了每次迭代的方法总数的趋势。图 15-12是匹配的代码覆盖率。这些示例说明了为什么需要在上下文中查看图表。如果您只看第一张显示方法数量的图表,您只会了解一半的情况。方法的数量在增加,看起来不错,但实际上覆盖率在下降。我们不知道覆盖率下降的原因,但这应该是问团队“为什么?”的触发因素。

Figures 15-12 and 15-13 are two examples of trends that work together. Figure 15-12 shows a trend of the total number of methods each iteration. Figure 15-12 is the matching code coverage. These examples show why graphs need to be looked at in context. If you only look at the first graph showing the number of methods, you’ll only get half the story. The number of methods is increasing, which looks good, but the coverage is actually decreasing. We do not know the reason for the decreased coverage, but it should be a trigger to ask the team, “Why?”

图 15-12方法数量趋势

Figure 15-12 Number of methods trend

图像

图 15-13测试覆盖率

Figure 15-13 Test coverage

图像

请记住,这些工具只能衡量您编写的代码的覆盖率。如果遗漏了某些功能,您的代码覆盖率报告将不会把它曝光。您的测试可能有 80% 的代码覆盖率,但您遗漏了本应拥有的 10% 的代码。用测试驱动开发有助于避免这个问题,但不要过度重视代码覆盖率统计数据。

Remember that these tools can only measure coverage of the code you’ve written. If some functionality was missed, your code coverage report will not bring that to light. You might have 80% code coverage with your tests, but you’re missing 10% of the code you should have. Driving development with tests helps avoid this problem, but don’t value code coverage statistics more than they deserve.

好的指标需要一些好的计划。额外的努力可以为您提供更有意义的数据。Pierre Veragen 的团队成员使用中断测试基线技术来了解他们的代码覆盖率指标是否有意义。他们手动在每种方法中引入缺陷,然后运行测试以确保测试能够捕获问题。一些测试只是确保代码返回一些值,任何值。Pierre 的团队确保测试返回正确的值。通过这种方式,他们可以确定他们的测试覆盖率是否足够好。

Good metrics require some good planning. Extra effort can give you more meaningful data. Pierre Veragen’s team members use a break-test baseline technique to learn if their code coverage metric is meaningful. They manually introduce a flaw into each method and then run their tests to make sure the tests catch the problem. Some tests just make sure the code returns some value, any value. Pierre’s team makes sure the tests return the correct value. In this way, they can determine whether their test coverage is good enough.

代码覆盖率只是难题的一小部分。就这样使用它。它不会告诉你你的测试有多好,但只有在测试期间运行了一段特定的代码。它也不会告诉您是否运行了应用程序的不同路径。了解您的应用程序并尝试确定您的最高风险区域,并为这些区域设定一个比低风险区域更高的覆盖目标。不要忘记将您的功能测试也包含在覆盖率报告中。

Code coverage is just one small part of the puzzle. Use it as such. It doesn’t tell you how good your tests are but only if a certain chunk of code was run during the test. It does not tell you if different paths through the application were run, either. Understand your application and try identifying your highest risk areas, and set a coverage goal that is higher for those areas than for low-risk areas. Don’t forget to include your functional tests in the coverage report as well.

缺陷指标

当您的团队设定与缺陷相关的目标时,使用适当的指标来衡量实现这些目标的进度。有些趋势是您希望针对整个版本监控的,有些是特定于迭代的。例如,如果您试图实现零缺陷,您可能希望在每次迭代结束时跟踪未解决的错误,或者在开发之后但在发布之前发现了多少错误。我们大多数人都想知道在代码投入生产后报告了多少缺陷,这是完全不同的事情。这些问题会在事后告诉您您的团队在上一个版本上的表现如何,但不会告诉您您在当前版本上的表现如何。他们可能会告诉您需要更改哪些流程以减少缺陷数量。Lisa 的团队更关心在新架构中重写的“新”代码中发现的生产缺陷。他们正在努力生产零缺陷的新代码,因此他们需要知道自己的表现如何。他们预计在遗留系统中会经常发现错误,其中只有最关键的功能被自动化的 GUI 冒烟测试覆盖,并且几乎没有自动化的单元测试和 GUI 背后的测试。

As your team sets goals related to defects, use appropriate metrics to measure progress toward those goals. There are trends that you will want to monitor for the whole release, and there are ones that are iteration-specific. For example, if you’re trying to achieve zero defects, you may want to track the open bugs at the end of each iteration, or how many bugs were found after development but before release. Most of us are interested in knowing how many defects have been reported after the code is in production, which is something completely different altogether. These issues will tell you after the fact how well your team did on the last release, but not how well you are doing on the current release. They may give you some indication of what processes you need to change to reduce the number of defects. Lisa’s team is more concerned with production defects found in the “new” code that was rewritten in the new architecture. They’re working hard to produce this new code with zero defects, so they need to know how well they’re doing. They expect that bugs will be found fairly often in the legacy system, where only the most critical functionality is covered by automated GUI smoke tests, and there are few automated unit and behind-the-GUI tests.

了解遗留代码的缺陷率可能是重构或重写它的充分理由,但团队的首要任务是做好新代码,因此他们按“新”和“旧”代码对错误进行分组,并专注于“新”错误。

Knowing the defect rate of legacy code might be good justification for refactoring or rewriting it, but the team’s top priority is doing a good job with the new code, so they group bugs by “new” and “old” code, and focus on the “new” bugs.

确保您的错误数据库可以跟踪您想要衡量的内容。您可能必须对数据库和流程进行一些更改才能获得所需的数据。例如,如果你想测量发布后在生产中发现了多少缺陷,你必须确保你有环境和版本作为必填字段,或者确保输入错误的人总是填写它们。

Make sure your bug database can track what you want to measure. You may have to make some changes in both the database and your process to get the data you need. For example, if you want to measure how many defects were found in production after a release, you have to make sure you have environment and version as mandatory fields, or make sure that people who enter bugs always fill them in.

有关缺陷跟踪系统的更多信息,请参阅第 5 章“转换传统流程”。

More on defect tracking systems can be found in Chapter 5, “Transitioning Traditional Processes.”

因为缺陷跟踪系统通常用于跟踪错误以外的目的,所以一定不要混淆数字。手动更新数据库的请求不一定反映现有代码的问题。正确使用您的缺陷跟踪工具以确保您的指标有意义。

Because defect tracking systems are often used for purposes besides tracking bugs, be sure not to muddle the numbers. A request for a manual update to the database doesn’t necessarily reflect an issue with the existing code. Use your defect tracking tool properly to ensure that your metrics are meaningful.

丽莎的故事

Lisa’s Story

定期评估您报告的指标,看看它们是否仍然相关。图 15-14显示了 Lisa 的团队多年来使用的两个缺陷报告。当我们第一次过渡到敏捷时,经理和其他人会查看这些报告以了解新流程带来的进展。四年后,我们的 ScrumMaster 发现没有人再阅读这些报告,所以我们停止制作它们。到那个时候,新缺陷的发生率已经急剧下降,没有人真正关心遗留代码中仍然存在的旧缺陷。

Periodically evaluate the metrics you’re reporting and see if they’re still relevant. Figure 15-14 shows two defect reports that Lisa’s team used for years. When we first transitioned to agile, managers and others looked at these reports to see the progress that resulted from the new process. Four years later, our ScrumMaster found that nobody was reading these reports anymore, so we quit producing them. By that time, rates of new defects had reduced dramatically, and nobody really cared about the old defects still hanging about in the legacy code.

——丽莎

—Lisa

图 15-14 Lisa 团队使用(和不再使用)的样本缺陷报告

Figure 15-14 Sample defect reports used (and no longer used) by Lisa’s team

图像

发布计划是评估您一直跟踪的指标的投资回报率的好时机。您花费了多少精力来收集和报告指标?他们会告诉您您需要知道的信息吗?您发布的代码是否符合您团队的内部质量标准?代码覆盖率在上升吗?团队是否实现了减少投入生产的缺陷数量的目标?如果没有,是否有充分的理由?

Release planning is a good time to evaluate the ROI of the metrics you’ve been tracking. How much effort are you spending to gather and report the metrics? Do they tell you what you need to know? Does the code you release meet your team’s standards for internal quality? Is the code coverage percentage going up? Is the team meeting its goals for reducing the number of defects that get out to production? If not, was there a good reason?

指标只是其中的一块拼图。使用您的发布、主题或项目规划会议,在业务需要时重新专注于交付业务价值。花一些时间了解您将要开发的功能。不要陷入对你的计划的承诺——情况一定会改变。相反,准备进行正确的活动并及时获得正确的资源以满足客户的优先事项。

Metrics are just one piece of the puzzle. Use your release, theme, or project planning meetings to refocus on delivering business value when the business needs it. Take some time to learn about the features you’re about to develop. Don’t get caught up with committing to your plans—the situation is bound to change. Instead, prepare for doing the right activities and getting the right resources in time to meet the customers’ priorities.

概括

Summary

当您的团队制定新主题或新版本的计划时,请牢记本章的要点。

As your team puts together its plan for a new theme or release, keep the main points of this chapter in mind.

图像在评估故事大小时,请考虑不同的观点,包括业务价值、风险、技术实施以及功能的使用方式。提出澄清问题,但不要陷入细节。

When sizing a story, consider different viewpoints, including business value, risk, technical implementation, and how the feature will be used. Ask clarifying questions, but don’t get bogged down in details.

图像测试人员可以通过功能集帮助确定“薄片”或“关键路径”,以帮助确定故事的优先级。如果高风险故事可能需要及早进行额外测试,请尽早安排。

Testers can help identify the “thin slice” or “critical path” through a feature set to help prioritize stories. Schedule high-risk stories early if they might require extra testing early.

图像故事测试工作的规模有助于确定该故事是否在发布范围内。

The size of testing effort for a story helps determine whether that story is in scope for the release.

图像测试人员可以帮助团队思考新故事将如何影响更大的系统。

Testers can help the team think about how new stories will impact the larger system.

图像当功能可能影响外部团队开发的系统或子系统时,计划额外的测试时间和资源。

Plan for extra testing time and resources when features may affect systems or subsystems developed by outside teams.

图像当团队确定发布范围时,评估测试范围并为其预算足够的时间和资源。

As the team identifies the scope of the release, evaluate the scope of testing and budget enough time and resources for it.

图像在发布计划期间花一些时间来解决基础设施、测试环境和测试数据问题。

Spend some time during release planning to address infrastructure, test environment, and test data concerns.

图像轻量级、敏捷的测试计划可以帮助确保在发布或项目的生命周期中解决所有测试注意事项。

A lightweight, agile test plan can help make sure all of the testing considerations are addressed during the life of the release or project.

图像考虑可能更适合您的团队的测试计划的替代方案;测试矩阵、电子表格甚至白板就足够了。

Consider alternatives to test plans that might be more appropriate for your team; test matrices, spreadsheets, or even a whiteboard may be sufficient.

图像 正式的发布计划可能不适合您的情况。在没有发布计划的情况下,考虑至少确定和讨论应该首先完成的前几个故事。

Formal release planning may not be appropriate for your situation. In the absence of release planning, consider identifying and discussing at least the first few stories that should be done first.

图像计划您希望在发布生命周期内捕获哪些指标;考虑您要解决的问题并仅捕获对您的团队有意义的指标。

Plan for what metrics you want to capture for the life of the release; think about what problem you are trying to solve and capture only those metrics that are meaningful for your team.

第16章开始行动

Chapter 16 Hit the Ground Running

图像

在敏捷开发中,我们通常喜欢“及时”地完成任务。我们看不到前方道路的弯道,所以我们专注于手头的活动。再一次,我们希望在开始每次新迭代时立即开始运行。这可能需要一些准备。烘焙在这里是一个很好的类比。你决定要烤饼干,因为有人过来了。在你开始之前,你要确保你有正确的成分。如果你不这样做,你要么去买你需要的东西,要么你选择另一种来制作。

In agile development, we generally like to do tasks “just in time.” We can’t see around the curves in the road ahead, so we focus on the activities at hand. Then again, we want to hit the ground running when we start each new iteration. That may require a little preparation. Baking is a good analogy here. You decide you want to bake cookies because someone is coming over. Before you start, you make sure you have the right ingredients. If you don’t, you either go buy what you need, or you choose a different kind to make.

不要做得太过分——如果迭代前的活动在迭代期间没有节省时间,或者帮助你做得更好,那么不要在迭代前做。做适合您团队的事情,并不断尝试。也许您会在迭代开始后进行其中的一些活动。这里有一些想法可以帮助您将质量“融入”到您的产品中。

Don’t go overboard—if a pre-iteration activity doesn’t save time during the iteration, or help you do a better job, don’t do it before the iteration. Do what is appropriate for your team, and keep experimenting. Maybe you’ll do some of these activities after the iteration starts instead. Here are some ideas to think about that might help you “bake quality in” to your product.

主动

Be Proactive

第 2 章“敏捷测试人员的十项原则”中,我们解释了敏捷测试人员必须如何转变他们的思维方式。我们没有等待工作来找我们,而是培养了一种积极主动的态度,我们起身去寻找方法贡献。随着敏捷开发快速而持续的步伐,很容易沉浸在当前迭代的故事中。我们忙于确保我们已经通过前期测试涵盖了这些功能,执行探索性测试以确保我们了解业务需求,并自动执行足够的回归测试,很难想到其他任何事情。然而,有时花点时间帮助我们的客户和我们的团队为下一次迭代做准备是合适的。当我们的团队即将开辟新天地,或处理复杂且有风险的故事时,在迭代之前进行一些工作可以帮助最大限度地提高团队的速度并最大限度地减少挫败感。

In Chapter 2, “Ten Principles of an Agile Tester,” we explained how agile testers have to shift their mind-set. Instead of waiting for work to come to us, we develop a proactive attitude where we get up and go look for ways to contribute. With the fast and constant pace of agile development, it’s easy to get immersed in the current iteration’s stories. We are so busy making sure we’ve covered the features with up-front tests, performing exploratory testing to be sure we’ve understood the business requirements, and automating adequate regression tests, it’s hard to think of anything else. However, it’s sometimes appropriate to take a bit of time to help our customers and our team prepare for the next iteration. When our team is about to break new ground, or work on complex and risky stories, some work before the iteration can help maximize our team’s velocity and minimize frustration.

我们肯定不想把所有时间都花在开会上,或计划可能重新确定优先级的故事。然而,如果我们能让我们的迭代计划进行得更快,并降低我们将要承担的故事的风险,那么在我们开始迭代之前进行一些研究和集思广益是值得的。

We sure don’t want to spend all our time in meetings, or planning for stories that might be re-prioritized. However, if we can make our iteration planning go faster, and reduce the risk of the stories we’re about to undertake, it’s worth doing some research and brainstorming before we start the iteration.

好处

Benefits

在迭代之前处理故事可能对分散在不同地理位置的团队特别有用。通过提前工作,有时间向每个人提供信息并让他们有机会提供意见。

Working on stories in advance of the iteration may be especially useful for teams that are split across different geographic locations. By working ahead, there’s time to get information to everyone and give them a chance to give their input.

问题是我们在每次短暂的敏捷迭代中都非常忙碌,很难抽出时间来讨论下一次迭代的故事,更不用说开始编写测试用例了。如果您的迭代总是顺利进行,故事递增地交付并且有足够的时间进行测试,并且交付的软件符合客户的期望,那么您可能不需要花时间提前准备。如果您的团队在完成故事时遇到困难,或者最终发现功能的实际行为与期望行为之间存在很大的不匹配,那么提前一点计划可能会在迭代期间为您节省时间。

The problem is that we’re so busy during each short agile iteration that it’s hard to find time to meet about the next iteration’s stories, much less start writing test cases. If your iterations always go smoothly, with stories delivered incrementally and plenty of time to test, and the delivered software matches customer expectations, you may not need to take time to prepare in advance. If your team has trouble finishing stories, or ends up with big mismatches between actual and desired behavior of features, a little advance planning may save you time during the iteration.

丽莎的故事

Lisa’s Story

我们的团队过去常常觉得我们没有时间提前计划下一次迭代。在多次误解故事并让故事远远超出估计,并发现大多数“错误”是遗漏的需求之后,我们决定在迭代中预算时间以开始讨论下一个。现在整个团队,包括产品负责人和其他需要的客户,在我们下一个冲刺的计划会议前一天会面一个小时或更短时间。

Our team used to feel we didn’t have time to plan in advance for the next iteration. After many experiences of misunderstanding stories and having them far exceed estimations, and finding most “bugs” were missed requirements, we decided to budget time in the iteration to start talking about the next one. Now the whole team, including the product owner and other customers as needed, meet for an hour or less the day before the planning meeting for our next sprint.

我们笑称这是“预先计划”会议。我们回顾下一次迭代的故事。产品负责人解释每个故事的目的。他在他的故事清单中检查了满意的商业条件和其他项目,并给出了期望行为的例子。我们就潜在的风险和依赖性进行头脑风暴,并在适当的时候确定钢线。

We laughingly call this the “pre-planning” meeting. We go over the stories for the next iteration. The product owner explains the purpose of each story. He goes over the business conditions of satisfaction and other items in his story checklists, and gives examples of desired behavior. We brainstorm about potential risks and dependencies, and identify steel threads where appropriate.

有时花几分钟听产品负责人对故事的解释就足够了。在其他时候,我们会花时间在白板上绘制故事的薄片。图 16-1显示了一个示例图,我们在其中详细了解了 UI 流和数据库表。注意“线程”的数字。线程 #1 是我们的关键路径。线程 #2 是第二层,依此类推。我们将这些图表的照片上传到 wiki,这样我们的远程开发人员也可以看到它们。

Sometimes it’s enough to spend a few minutes listening to the product owner’s explanation of the stories. At other times, we take time to diagram thin slices of a story on the whiteboard. Figure 16-1 shows an example diagram where we got into both the details of the UI flow and the database tables. Note the numbers for the “threads.” Thread #1 is our critical path. Thread #2 is the second layer, and so on. We upload photos of these diagrams to the wiki so our remote developer can see them too.

图 16-1示例规划白板图

Figure 16-1 Sample planning whiteboard diagram

图像

我们可以开始考虑第二天我们可以写什么任务卡,以及我们可以对每个故事采取什么方法。出于某种原因,能够在一夜之间反复思考这些故事会让我们在第二天的实际迭代计划会议上更有效率。这样做几次迭代后,我们花在规划迭代上的时间总体上减少了,尽管我们有两次会议来做这件事。

We can start thinking about what task cards we might write the next day and what approach we might take to each story. For some reason, being able to ruminate about the stories overnight makes us more productive in the actual iteration planning meeting the next day. After doing this for a few iterations, we were spending less time overall in planning the iteration, even though we had two meetings to do it.

有时我们会邀请其他客户讨论直接影响他们的故事。如果他们当时不可用,我们在迭代计划会议之前还有时间与他们交谈以澄清故事。

Sometimes we pull in other customers to discuss stories that affect them directly. If they aren’t available right then, we still have time before our iteration planning meeting to talk to them to clarify the story.

在一次前期策划会上,我们的产品负责人介绍了一个获取共同基金业绩数据的故事。我们会向供应商发送一个包含共同基金列表的文件,供应商会在网站上提供一个 XML 文件,其中包含这些基金的所有最新业绩信息。然后我们会将该数据上传到我们的数据库中。

In one pre-planning meeting, our product owner introduced a story about obtaining performance data for mutual funds. We would send a file to the vendor containing a list of mutual funds, and the vendor would provide an XML file on a website with all the latest performance information for those funds. We would then upload that data into our database.

在前期计划会议中,我们提出了诸如“我们发送给供应商的文件的格式是什么?”之类的问题。“每个基金的‘截至’日期总是该月的最后一天吗?” “包含 XML 的网站是否有任何安全措施?” “我们是否会获得具有新数据的同一基金和“截至”日期的记录,或者我们是否可以忽略数据库中已有日期的记录?”

In the pre-planning meeting, we asked questions such as, “What’s the format of the file we send the vendor?” “Is the ‘as-of’ date for each fund always the last day of the month?” “Is there any security on the website that contains the XML?” “Will we ever get a record for the same fund and “as-of” date that has new data, or can we ignore records with a date we already have in our database?”

到第二天的迭代计划会议时,产品负责人已经得到了我们所有问题的答案。编写任务卡进行得很快,编码可以在正确的假设下进行。

By the next day’s iteration planning meeting, the product owner had obtained answers to all our questions. Writing task cards went quickly, and coding could proceed with correct assumptions.

通常,当我们在预迭代计划讨论中讨论故事时,我们会找到一个更简单的故事解决方案。我们发现要走得快,我们需要先放慢速度。

Often, we find a much simpler solution to a story when we discuss it in the pre-iteration planning discussion. We found that to go fast, we needed to slow down first.

——丽莎

—Lisa

你真的需要这个吗?

Do You Really Need This?

您的团队可能不需要太多或任何提前准备。Pierre Veragen 和 Erika Boyer 向 Lisa 描述了他们在 iLevel by Weyerhaeuser 的团队如何在他们的迭代启动会议上一起编写用户验收测试。

Your team may not need much or any advance preparation. Pierre Veragen and Erika Boyer described to Lisa how their teams at iLevel by Weyerhaeuser write user acceptance tests together at their iteration kickoff meeting.

这些测试与故事叙述一起写在维基页面或类似工具上,稍后在团队成员为每个故事编写任务卡并开始编写更多测试和代码时使用。例子变成了可执行的测试。因为随着团队对故事的了解越来越多,测试也会发生变化,所以团队可能会选择不维护开始时编写的原始测试。开始时保持简单,稍后再深入研究细节。

These tests, which were written on a wiki page or some similar tool along with the story narrative, are used later when team members write task cards for each story and start writing more tests and code. Examples are turned into executable tests. Because the tests change as the team learns more about the story, the team may opt not to maintain the original ones that were written at the start. Keep it simple to start with, and dig into details later.

丽莎随后观察了他们的一次计划会议,并亲眼目睹了这项技术的有效性。即使产品经理提供了具体的示例,将它们变成测试也可能会消除遗漏的需求。他们的团队在迭代计划会议之前不需要这样做,但并非所有团队都是这样。

Lisa subsequently observed one of their planning sessions and saw first-hand how effective this technique was. Even when the product manager provides concrete examples, turning them into tests may flush out missing requirements. Their team did not need to do this before the iteration planning session, but it is not the case with all teams.

丽莎的故事

Lisa’s Story

我的团队喜欢一起编写测试的做法,但是因为我们在迭代计划期间编写任务卡,所以我们决定在预计划会议期间一起编写用户验收测试。我们发现这使我们的讨论更加集中,并且我们可以更快地理解每个故事。我们在准确交付客户心中所想方面也做得更好。我们的客户注意到质量上的差异,我们的产品负责人鼓励我们继续这种做法。

My team liked the practice of writing tests together, but because we were writing task cards during iteration planning, we decided to write user acceptance tests together during the pre-planning meeting. We found this kept our discussions focused and we understood each story more quickly. We also did a better job of delivering exactly what the customer had in mind. Our customers noticed a difference in quality, and our product owner encouraged us to continue this practice.

——丽莎

—Lisa

尝试进行简短的预迭代讨论和测试写作会议。您需要多次迭代才能找到团队的节奏,并了解提前讨论故事是否能让您在迭代期间更有效率。

Experiment with short pre-iteration discussions and test-writing sessions. It’ll take you several iterations to find your team’s rhythm, and find out if advance story discussions make you more productive during the iteration.

提前准备的潜在缺点

Potential Downsides to Advance Preparation

“提前工作”存在风险。您可以花时间了解有关某个功能的更多详细信息,结果业务人员在最后一刻重新确定优先级并无限期地推迟该功能。在适当的时候投入准备时间。当你知道你有一个复杂的主题或故事即将到来,并且它有一个严格的截止日期,就像 Lisa 的团队对声明故事所做的那样,考虑花一些时间预先检查不同的观点。提前讨论故事的唯一原因是为了在迭代计划和开发过程中节省时间。更深入地了解功能行为可以加快测试和编码速度,并有助于确保您提供正确的功能。

There’s a risk to “working ahead.” You could spend time learning more details about a feature only to have the business people re-prioritize at the last minute and put that feature off indefinitely. Invest preparation time when it’s appropriate. When you know you have a complex theme or story coming up, and it has a hard deadline such as Lisa’s team had with the statement story, consider spending some time up front checking out different viewpoints. The only reason to discuss stories in advance is to save time during iteration planning and during development. A deeper understanding of the feature behavior can speed up testing and coding, and can help make sure you deliver the right functionality.

如果您的情况如此多变以至于故事可能会在迭代开始的那一天重新确定优先级,那么不值得尝试进行此计划。相反,请确保在计划会议期间为这些讨论预留时间。

If your situation is so dynamic that stories might be re-prioritized the day that the iteration starts, it isn’t worth trying to do this planning. Instead, make sure you budget time for these discussions during your planning meeting.

提前澄清

Advance Clarity

Lisa 的产品负责人 Steve Perkins 提出了“预先清晰度”这个词。每个组织的不同部分有不同的优先事项和议程。例如,业务开发部正在寻找新功能来吸引新业务,而运营部则优先考虑可以减少用户来电次数的功能。开发团队试图了解业务需求的范围并了解每个人的工作。

Lisa’s product owner, Steve Perkins, came up with the term “advance clarity.” Different parts of each organization have different priorities and agendas. For example, Business Development is looking for new features to attract new business, while Operations is prioritizing features that would reduce the number of phone calls from users. The development team tries to understand the range of business needs and get a feel for each individual’s job.

对于许多不同的议程,需要有人来决定在下一次迭代中应该实施哪些故事。因为有很多方法可以实现任何给定的故事,所以必须有人决定具体的需求,并以示例、满足条件和测试用例的形式捕获它们。史蒂夫让每个人都聚在一起,就他们希望从每个故事中获得的价值达成一致,并提供“预先的清晰度”。

With many different agendas, someone needs to decide what stories should be implemented in the next iteration. Because there are many ways to implement any given story, someone has to decide the specific requirements and capture them in the form of examples, conditions of satisfaction, and test cases. Steve gets everyone together to agree on the value they want from each story, and to provide “advance clarity.”

客户用一种声音说话

Customers Speak with One Voice

Scrum 提供了产品负责人的有用角色,以帮助所有客户“用同一个声音说话”。无论你是否在 Scrum 团队中,找到一些方法来帮助你的客户就故事的优先级以及如何每个故事的组成部分都应该被实现。管理层的支持是至关重要的,因为担任此角色的任何人都需要时间和权力来让所有人达成共识。

Scrum provides the helpful role of the product owner to help all the customers “Speak with One Voice.” Whether or not you’re on a Scrum team, find some way to help your customers agree on the priority of the stories and how the components of each story ought to be implemented. Management support is crucial, because any person in this role needs time and authority to get everyone on the same page.

其他团队使用业务分析师在下一次迭代之前帮助充实故事。在 Janet 工作的一个组织中,客户无法全职回答问题,但每个团队都有一名业务分析师与客户合作,在迭代计划会议之前充实需求。如果有任何她在会议上无法回答的问题,团队要么直接给客户打电话,要么分析师会后立即跟进。

Other teams use business analysts to help flesh out the stories before the next iteration. In one organization Janet worked with, the customers were not available full-time to answer questions, but each team had a business analyst that worked with the customers to flesh out the requirements before the iteration planning meeting. If there were any questions that she could not answer at the meeting, the team either called the customer directly or the analyst followed up immediately after the meeting.

作为一名测试人员,您想参加故事编写和优先级确定会议。提出有助于客户专注于核心功能的问题,即他们需要的关键业务价值。帮助参与者将注意力集中在具体的例子上,这些例子使故事的含义具体化。在涉及多个客户的会议中,拥有强大的协调人和确定共识的方法至关重要。

As a tester, you want to sit in on story writing and prioritization meetings. Ask questions that help the customers focus on the core functionality, the critical business value they need. Help participants stay focused on concrete examples that crystallize the meaning of the stories. In meetings that involve multiple customers, it is critical to have a strong facilitator and a method for determining consensus.

与代码一样,故事只有最少的内容才是最好的。例如,互联网购物车需要一些方法来删除不需要的物品,但是将物品从购物车移动到“稍后保存”列表的能力可能需要等待。在迭代之前讨论这个可能会有所帮助,这样团队就清楚需要计划哪些任务。首先关注最简单的事情,并用一个例子来说明。

As with code, stories are best if they have the bare minimum. For example, an Internet shopping cart needs some way to delete unwanted items, but the ability to move items from the cart to a “save for later” list can probably wait. It may be helpful to talk about this before the iteration, so that the team is clear on what tasks need to be planned. Focus on the simplest thing first and use an example to make it clear.

获取所有观点

从不同的客户那里获取对一个故事的需求,每个客户都有不同的议程,可能会造成混乱。这就是为什么客户团队中的某个人必须达成共识并协调所有观点。这并不意味着我们不应该从不同的客户那里获得意见。作为测试人员,您要从多个角度考虑每个故事。它有助于了解故事对不同角色的人意味着什么。

Getting requirements from different customers for a story, each of whom has a different agenda, might create chaos. That’s why it’s essential for someone on the customer team to get consensus and coordinate all points of view. This doesn’t mean we shouldn’t get input from different customers. As a tester, you’re considering each story from multiple points of view. It helps to know what the story means to people in different roles.

丽莎的故事

Lisa’s Story

当我的公司决定重新设计退休计划参与者的季度账户报表时,业务方面的不同人员出于不同的原因希望进行更改。计划管理员想要一个清晰易懂的布局,以最大限度地减少困惑的参与者拨打客户支持电话的次数。

When my company decided to redesign the retirement plan participants’ quarterly account statements, different people on the business side wanted changes for different reasons. The plan administrators wanted a clearly understandable layout that would minimize the number of calls from confused participants to customer support.

例如,他们希望声明显示参与者最近捐款的日期和金额。这有助于参与者知道她是否雇主迟迟未将供款记入账目。业务开发人员需要可以向潜在客户销售的时髦新功能,例如按基金类别划分的绩效图表。我们的法人需要声明中的一些新文本和数据以满足联邦法规。

For example, they wanted the statement to show the date and amount of the participant’s most recent contribution. This helps the participant know whether her employer is late in posting contributions to the accounts. Business development wanted jazzy new features that they could sell to potential customers, such as graphs of performance by fund category. Our legal person needed some new text and data on the statement to satisfy federal regulations.

虽然产品所有者平衡了所有不同的需求并呈现了最终的声明布局,但对于我们的团队来说,了解每条新信息背后的目的仍然很重要。我们需要直接与计划管理、业务开发和法律领域的业务专家以及产品负责人交谈。一名测试人员和一名程序员与每个小组会面以收集不同的观点。通过在开始故事以收集和显示数据之前执行此操作,我们可以更清楚地了解需求,甚至提出建议以更有效地生成信息。

While the product owner balanced all the different needs and presented the final statement layout, it was still important for our team to understand the purpose behind each new piece of information. We needed to talk directly to business experts in the plan administration, business development, and legal areas, and to the product owner. A tester and a programmer met with each group to gather the different viewpoints. By doing this before starting on stories to gather and display data, we understood the requirements much more clearly and even made suggestions to produce the information more efficiently.

——丽莎

—Lisa

确保您尽可能高效地收集这些数据。有时整个团队了解需求很重要,有时一两个团队成员进行研究就足够了。

Make sure you are as efficient as possible in collecting this data. Sometimes it is important for the whole team to understand the need, and sometimes it is sufficient for one or two of the team members to do the research.

故事大小

Story Size

当您与客户团队成员讨论下一次迭代的故事时,提出问题以帮助他们确保每个故事都能提供所需的价值。这是确定他们可能需要写的新故事的好时机。即使团队之前确定了故事的大小,您也可能会发现故事比之前想象的要大。您甚至可能会发现某个功能的实现可能比计划的更简单,故事也可能更小。

As you discuss stories for the next iteration with the customer team members, ask questions to help them make sure each story delivers the value needed. This is a good time to identify new stories they might need to write. Even though the team sized the stories previously, you might find a story is bigger than previously thought. You might even discover that a feature can be implemented more simply than planned, and the story can be smaller.

有时,当故事规模大时,会做出假设,但经过进一步调查,结果证明是错误的。即使是简单的故事也值得仔细研究。任何人都很难记住一份申请的所有细节。

Sometimes assumptions are made when the story is sized and on further investigation turn out to be false. Even simple stories deserve a closer look. It’s hard for any one person to remember all the details of an application.

丽莎的故事

Lisa’s Story

这里有一些故事的例子,结果证明比原先想象的要大得多或小得多。

Here are some examples of stories that turned out to be significantly bigger or smaller than originally thought.

1.故事是为给定公司退休计划的所有参与者生成一份账户报表文件,该文件将发送给供应商,由供应商打印和邮寄报表。它最初的大小是假设所有的陈述都是恰好三页长的。经过进一步调查,我们发现一些参与者有四页的陈述,但供应商要求所有陈述的长度相同。我们的业务专家必须决定是否要有一个功能来标记参与者有四页声明的任何计划并手动处理这些计划,或者更改声明以使其全部为四页。这比原来的努力要大得多故事。在我们开始开发故事后,客户提出了另一个要求:如果任何参与者的地址丢失或无效,则应将声明邮寄给雇主。这是合理的,但我们在评估故事大小时并不知道这一点。

1. The story was to produce a file of account statements for all participants in a given company retirement plan, which was to be sent to a vendor who would print and mail the statements. It was originally sized with the assumption that all statements were exactly three pages long. Upon further investigation, we discovered that some participants had four-page statements, but the vendor required that all statements be the same length. Our business experts had to decide whether to have a feature to flag any plans whose participants had four-page statements and deal with those manually, or change the statements to make them all four pages long. That’s a much bigger effort than the original story. After we started developing the story, the customers revealed another requirement: If any participant’s address was missing or invalid, the statement should be mailed to the employer instead. It’s reasonable, but we didn’t know about it when we sized the story.

2.我们的客户希望开始在 UI 的不同位置显示销售电话号码。每个合作伙伴站点都有不同的销售电话号码,当时大约有 25 个不同的合作伙伴站点。这听起来像是一个直截了当的故事,甚至没有向团队说明规模。开发经理只是给它分配了一个小点值,它只是被“添加”到迭代中。他假设电话号码存储在数据库中,但实际上它是硬编码在每个合作伙伴的“联系人”页面的 HTML 中。在数据库中为每个合作伙伴存储正确的数字,并更改代码以检索值,使故事变大了一倍,并且在该迭代中没有空间容纳它,因此没有完成。

2. Our customers wanted to start displaying the sales phone number in various locations in the UI. There is a different sales phone number for each partner’s site, and at the time there were about 25 different partner sites. This sounded like such a straightforward story that it wasn’t even given to the team to size. The development manager just assigned it a small point value, and it was just “added” to the iteration. He had assumed the phone number was stored in the database, when in fact it was hard-coded in the HTML of each partner’s “contact” page. Storing the correct number for each partner in the database, and changing the code to retrieve the value, made the story twice as big, and there wasn’t room for it in that iteration, so it did not get done.

3.我们为用户界面调整了故事大小,以允许管理员提交批处理作业请求,以重新平衡满足特定条件的参与者帐户。它包括一个确认页面,显示受影响的参与者数量。因为请求被排队以作为异步批处理作业运行,所以确定哪些参与者受到影响的代码在批处理作业的代码中。重构代码以获取请求时的参与者数量是一项艰巨的工作。在我们开始处理这个故事后,我们在提交请求时询问了该功能的主要用户他是否真的需要这个号码,他认为没有必要。这个故事变得比原先想象的要小得多。我们总是提出问题以找出客户想要的真正商业价值,并消除没有良好投资回报率的组件。

3. We sized a story for the user interface to allow administrators to submit a request for a batch job to rebalance participant accounts that met a certain condition. It included a confirmation page displaying the number of participants affected. Because the request was queued to run as an asynchronous batch job, the code to determine which participants were affected was in the batch job’s code. Refactoring the code to obtain the number of participants at request time was a big job. After we started working on the story, we asked the primary user of the feature whether he really needed that number upon submitting the request, and he decided it wasn’t necessary. The story became much smaller than originally thought. We always ask questions to find out the true business value that the customers want and eliminate components that don’t have a good ROI.

——丽莎

—Lisa

这些故事表明,预先提出几个问题可能会在迭代期间节省时间,而这些时间可以用来弄清楚如何处理新发现。然而,我们认识到并非所有发现都能及早发现。例如,在第一个故事中,一个关于语句大小的简单问题可能避免了最后一刻关于如何处理四页语句的混淆,但在编码或测试之前可能没有考虑不准确的地址问题。

These stories show that a few questions up front might save time during the iteration that could be spent figuring out what to do with new discoveries. However, we recognize that not all discoveries can be found early. For example, on the first story, a simple question about statement size may have prevented last-minute confusion about how to handle four-page statements, but the inaccurate address issue may not have been considered until it was being coded or tested.

我们知道一路上总会有发现,但如果我们能首先抓住大的“陷阱”,那将有助于团队尽可能有效地工作。

We know there will always be discoveries along the way, but if we can catch the big “gotchas” first, that will help the team work as effectively as possible.

地域分散的团队

Geographically Dispersed Teams

为下一次迭代做一些准备可能对分散在不同地点的团队有用。分布在多个地点的团队可以通过电话会议、在线会议或电话会议来进行迭代计划。Lisa 的一个团队使用的一种做法是给每个团队分配一个即将发生的故事的子集,并让他们提前写任务卡。在计划会议期间,每个人都可以查看所有任务卡并根据需要进行更改。前期工作加强了沟通,使故事和任务对每个人都可见,并加快了规划过程。

Some preparation for the next iteration may be useful for teams that are split across different locations. Teams that are distributed in multiple locations may do their iteration planning by conference call, online meeting, or teleconference. One practice, which a team of Lisa’s used, is to assign each team a subset of the upcoming stories and have them write task cards in advance. During the planning meeting, everyone can review all the task cards and make changes as needed. The up-front work enhances communication, makes the stories and tasks visible to everyone, and speeds up the planning process.

当然,这是假设团队正在使用电子故事板或任务板。Lisa 的团队使用 Thoughtwork 的 Mingle,但还有许多其他产品可用于此目的。

Of course, this assumes that the team is using an electronic story or task board. Lisa’s team uses Thoughtwork’s Mingle, but there are many other products out there that serve this purpose.

如果客户不能随时回答问题和做出决定,则应授权随时可以访问的其他领域专家通过确定优先级和表达所需系统来指导团队行为与例子。测试人员和业务分析师经常被要求进行这些活动。

If customers aren’t readily available to answer questions and make decisions, other domain experts who are accessible at all times should be empowered to guide the team by determining priorities and expressing desired system behavior with examples. Testers and business analysts are often called upon to do these activities.

例子

Examples

您可能会注意到,我们几乎在本书的每一章中都讨论了示例。示例是了解和说明所需(和不需要)功能的有效方式;在整个开发周期中使用它们是值得的。我们的座右铭是布赖恩·马里克 (Brian Marick) 创造的:“一个例子现在就派上用场了。” (参见图 16-2。)用一个真实的例子开始你对特征和故事的讨论。这个想法已经流行起来,因此在最近的功能测试研讨会上,我们讨论了将其称为“示例驱动开发”的想法。

You may notice that we talk about examples in just about every chapter of this book. Examples are an effective way to learn about and illustrate desired (and undesired) functionality; it’s worth using them throughout your development cycle. Our motto was coined by Brian Marick: “An example would be handy right about now.” (See Figure 16-2.) Start your discussions about features and stories with a realistic example. The idea has taken off, so that at a recent workshop for functional testing we were discussing ideas around calling it “Example-Driven Development.”

图 16-2 Brian Marick 的示例贴纸

Figure 16-2 Brian Marick’s example sticker

图像

当 Lisa 的团队成员与他们的产品负责人会面讨论下一个迭代时,他们要求他提供每个故事所需行为的示例。这使讨论保持在一个具体的层面上,并且是了解新功能应该如何工作的快速方法。执行此操作时手边有一块白板,然后开始绘图。如果一些团队成员在很远的地方,可以考虑使用允许每个人都看到白板图并参与讨论的工具。与您的客户或他们的代理人一起研究真实的例子。在发布计划期间,考虑不同的观点:业务、最终用户、开发人员和业务合作伙伴。与发布计划不同,您正在查看更多细节,因为这些是您为下一次迭代计划的故事。

When Lisa’s team members meet with their product owner to talk about the next iteration, they ask him for examples of desired behavior for each story. This keeps the discussion at a concrete level and is a fast way to learn how the new features should work. Have a whiteboard handy while you do this, and start drawing. If some team members are in a distant location, consider using tools that allow everyone to see whiteboard diagrams and participate in the discussion. Go through real examples with your customers or their proxies. As during release planning, consider different points of view: the business, end users, developers, and business partners. Unlike release planning, you are looking at far more detail because these are the stories you are planning for the next iteration.

使用示例,您可以编写高级测试来充实每个故事。您可能不需要在迭代开始之前执行此操作,但对于复杂的故事,提前编写至少一个快乐路径和一个消极路径测试用例可能是个好主意。让我们考虑一下图 16-3中的故事。

Using examples, you can write high-level tests to flesh out each story a bit more. You may not need to do this before the iteration starts, but for complex stories, it can be a good idea to write at least one happy path and one negative path test case in advance. Let’s consider the story in Figure 16-3.

图 16-3从购物车中删除商品的 Story

Figure 16-3 Story for deleting items from shopping cart

图像

产品负责人在白板上勾勒出所需的 UI。每个项目旁边都有一个“删除”复选框和一个“更新购物车”按钮。用户可以选择一个或多个项目并单击按钮删除项目。高级测试可能是:

The product owner sketches out the desired UI on the whiteboard. There’s a “delete” checkbox next to each item and an “update cart” button. The user can select one or more items and click the button to remove the items. The high-level tests might be:

图像当用户点击项目旁边的删除复选框并点击“更新购物车”按钮时,页面刷新显示该项目不再在购物车中。

When the user clicks the delete checkbox next to the item and clicks the “update cart” button, the page refreshes showing the item is no longer in the cart.

图像当用户单击购物车中每个项目旁边的删除复选框并单击“更新购物车”按钮时,页面刷新显示一个空购物车。(这会产生问题——用户应该被引导到另一个页面吗?应该显示“继续购物”按钮吗?)

When the user clicks the delete checkboxes next to every item in the cart and clicks the “update cart” button, the page refreshes showing an empty cart. (This will generate questions—should the user be directed to another page? Should a “keep shopping” button display?)

图像当用户单击“更新购物车”按钮而没有选中要删除的项目时,页面会刷新并且不会从购物车中删除任何内容。

When the user clicks the “update cart” button without checking an item for delete, the page is refreshed and nothing is removed from the cart.

要求您的客户在迭代之前写下示例和高级测试用例。这可以帮助他们更多地思考故事并帮助定义他们的满意度条件。它还可以帮助他们确定哪些功能是关键的,哪些可以等待。它还有助于定义故事何时完成并管理团队之间的期望。

Ask your customers to write down examples and high-level test cases before the iteration. This can help them think through the stories more and help define their conditions of satisfaction. It also helps them identify which features are critical, and which might be able to wait. It also helps to define when the story is done and manage expectations among the team.

图 16-4显示了一个示例模型,其中产品所有者在现有页面上标记了更改。使用旧系统的现有屏幕截图时要小心,因为您将冒着让新系统看起来与旧系统完全一样的风险,即使这不是您想要的。

Figure 16-4 shows a sample mock-up, where the product owner marked changes on the existing page. Be careful about using an existing screenshot from an old system, because you will run the risk of having a new system look exactly like the old one even if that is not what you wanted.

图 16-4示例客户模型

Figure 16-4 Sample customer mock-up

图像

模型对于涉及 UI 或报告的故事至关重要。请您的客户就页面的外观提出他们的想法。与团队分享这些想法。一种想法是扫描它们并将它们上传到 wiki,这样每个人都可以访问。使用这些作为起点并制作更多纸质原型,或将它们画在白板上。这些可以拍照并上传以供远程团队成员查看。

Mock-ups are essential for stories involving the UI or a report. Ask your customers to draw up their ideas about how the page should look. Share these ideas with the team. One idea is to scan them in and upload them on the wiki so everyone has access. Use those as a starting point and do more paper prototypes, or draw them on the whiteboard. These can be photographed and uploaded for remote team members to see.

测试策略

Test Strategies

当您了解下一次迭代的故事时,请考虑如何接近测试它们。他们是否提出了任何特殊的自动化挑战?是否需要任何新工具?

As you learn about the stories for the next iteration, think about how to approach testing them. Do they present any special automation challenge? Are any new tools needed?

丽莎的故事

Lisa’s Story

最近,我公司需要更换语音应答单元硬件和交互式语音接口软件。承包商将提供操作语音应用程序的软件,但它需要通过存储过程与数据库进行交互。

Recently, our company needed to replace the voice response unit hardware and interactive voice interface software. A contractor was to provide the software to operate the voice application, but it needed to interact via stored procedures with the database.

这与我们之前使用过的任何软件都有很大的不同,因此在涉及与该项目相关的故事的第一次迭代计划之前,多花一天时间研究其他团队如何测试此类应用程序是很有帮助的。在迭代计划会议期间,我们能够编写与所需测试相关的任务并给出更好的估计。

This was a big departure from any software we’d worked on before, so it was helpful to have an extra day to research how other teams have tested this type of application before the first iteration planning that involved a story related to this project. During the iteration planning session, we were able to write tasks that were pertinent to the testing needed and give better estimates.

——丽莎

—Lisa

当您的团队着手开发一种新型软件时,您可能会决定进行一次开发高峰,看看您可以从中学到什么来开发它。同时,尝试测试尖峰以帮助确保您知道如何通过测试推动开发以及如何测试生成的软件。如果即将出现一个重要的新史诗或功能,写一些卡片来研究它并提前一两次举行头脑风暴会议。这有助于您在实际开始编码时了解要计划哪些故事和任务。一个想法是拥有一个“侦察”团队,看看哪些技术解决方案可能适用于即将到来的故事或主题。

When your team embarks on a new type of software, you may decide to do a development spike to see what you can learn about how to develop it. At the same time, try a test spike to help make sure you’ll know how to drive the development with tests and how to test the resulting software. If a major new epic or feature is coming up, write some cards to research it and hold brainstorming meetings an iteration or two in advance. That helps you know what stories and tasks to plan when you actually start coding. One idea is to have a “scout” team that looks at what technical solutions might work for upcoming stories or themes.

确定缺陷的优先​​级

Prioritize Defects

在我们的理想世界中,我们希望在每次迭代结束时零缺陷,并且绝对是在发布结束时。然而,我们认识到我们并不生活在一个理想的世界中。有时我们需要担心遗留系统缺陷,有时修复缺陷的价值不足以让企业修复。这些缺陷会发生什么?我们将在第 18 章编码和测试”中讨论策略,但现在,我们只考虑我们有缺陷要处理。

In our ideal world, we want zero defects at the end of each iteration and definitely at the end of the release. However, we recognize that we don’t live in an ideal world. Sometimes we have legacy system defects to worry about, and sometimes fixing a defect is just not high enough value for the business to fix. What happens to these defects? We’ll talk about strategies in Chapter 18, “Coding and Testing,” but for now, let’s just consider that we have defects to deal with.

在下一次迭代之前,最好是与客户一起审查未解决的问题,并对修复与将它们留在系统中的价值进行分类。那些被认为有必要修复的应该安排到下一个迭代中。

Before the next iteration is an ideal time to review outstanding issues with the customer and triage the value of fixing versus leaving them in the system. Those that are deemed necessary to be fixed should be scheduled into the next iteration.

资源

Resources

在迭代之前要仔细检查的另一件事是您的团队是否拥有完成任何高风险故事所需的所有资源。您需要与其他项目共享的专家吗?例如,如果其中一个故事存在安全风险或涉及安全功能,您可能需要一名安全专家。如果要进行负载测试,您可能需要有一个特殊的工具,或者需要来自另一个团队的负载测试专家,甚至是提供负载测试服务的供应商的帮助。这是您提前计划的最后机会。

Another thing to double-check before the iteration is whether your team has all the resources you need to complete any high-risk stories. Do you need any experts who are shared with other projects? For example, you may need a security expert if one of the stories poses a security risk or is for a security feature. If load testing will be done, you may need to have a special tool, or have help from a load testing specialist from another team, or even a vendor who provides load testing services. This is your last chance to plan ahead.

概括

Summary

您的团队可能需要也可能不需要在迭代之前做任何准备。因为优先级在敏捷开发中变化很快,所以您不想浪费时间计划可能被推迟的故事。但是,如果您即将实施一些新技术,开始一个复杂的新主题,希望在迭代计划中节省时间,或者您的团队被分配到不同的地点,您可能会发现一些前期计划和研究会很有成效. 作为测试人员,您可以执行以下操作:

Your team may or may not need to do any preparation in advance of an iteration. Because priorities change fast in agile development, you don’t want to waste time planning stories that may be postponed. However, if you’re about to implement some new technology, embark on a complex new theme, hope to save time in iteration planning, or your team is divided into different locations, you might find some up-front planning and research to be productive. As a tester, you can do the following:

图像通过提出问题和获取示例,帮助客户实现“提前清晰”——就每个故事的预期行为达成共识。

Help the customers achieve “advance clarity”—consensus on the desired behavior of each story—by asking questions and getting examples.

图像积极主动,在迭代之前了解复杂的故事,并确保它们的大小正确。

Be proactive, learn about complex stories in advance of the iteration, and make sure they’re sized correctly.

图像您并不总是需要提前准备才能在下一次迭代中开始运行。不要做任何不能在迭代期间节省时间或确保更成功地满足客户需求的准备工作。

You don’t always need advance preparation to be able to hit the ground running in the next iteration. Don’t do any preparation that doesn’t save time during the iteration or ensure more success at meeting customer requirements.

图像协调不同地点之间的关系,促进沟通。有很多工具可以帮助解决这个问题。

Coordinate between different locations and facilitate communication. There are many tools to help with this.

图像获取示例以帮助说明每个故事。

Obtain examples to help illustrate each story.

图像在下一次迭代之前为新的和不寻常的功能制定测试策略。

Develop test strategies in advance of the next iteration for new and unusual features.

图像对现有缺陷进行分类并确定其优先级,以确定是否应为下一次迭代安排任何缺陷。

Triage and prioritize existing defects to determine whether any should be scheduled for the next iteration.

图像确定是否需要为下一次迭代安排当前不在手边的任何必要测试资源。

Determine whether any necessary testing resources not currently at hand need to be lined up for the next iteration.

第 17 章迭代启动

Chapter 17 Iteration Kickoff

图像

敏捷测试人员在迭代计划中扮演着重要的角色,帮助计划测试和开发任务。随着迭代的进行,测试人员积极与客户和开发人员合作,编写有助于指导开发的高级测试,引出和说明示例,确保故事是可测试的。让我们仔细看看敏捷测试人员在每次迭代开始时的活动

Agile testers play an essential role during iteration planning, helping to plan testing and development tasks. As the iteration gets under way, testers actively collaborate with customers and developers, writing the high-level tests that help guide development, eliciting and illustrating examples, making sure stories are testable. Let’s take a closer look at the agile tester’s activities at the beginning of each iteration.

迭代计划

Iteration Planning

大多数团队通过计划会议开始他们的新迭代。在此之前可能会进行回顾或“经验教训”会议,以回顾在上一次迭代中哪些工作良好,哪些工作不顺利。尽管回顾的行动项目或“开始、停止、继续”建议会影响即将开始的迭代,但我们将在第 19 章“结束迭代”中将回顾作为迭代结束的活动讨论

Most teams kick off their new iteration with a planning session. This might be preceded by a retrospective, or “lessons learned” session, to look back to see what worked well and what didn’t in the previous iteration. Although the retrospective’s action items or “start, stop, continue” suggestions will affect the iteration that’s about to start, we’ll talk about the retrospective as an end-of-iteration activity in Chapter 19, “Wrap Up the Iteration.”

在为迭代计划工作时,开发团队一次讨论一个故事,编写和估算实施所需的所有任务那个故事。如果您提前做了一些工作来为迭代做准备,那么这个计划会议可能会进行得相当快。

While planning the work for the iteration, the development team discusses one story at a time, writing and estimating all of the tasks needed to implement that story. If you’ve done some work ahead of time to prepare for the iteration, this planning session will likely go fairly quickly.

刚接触敏捷开发的团队通常需要大量时间来进行迭代计划会议。Lisa 的团队刚开始时,迭代计划通常需要一整天。现在他们在两三个小时内完成,其中包括回顾的时间。Lisa 的团队使用投影仪来显示他们 wiki 中的用户验收测试用例和满意度条件,以便团队中的每个人都可以看到。他们还投射了他们的在线故事板工具,他们在其中编写任务卡。他们计划会议的另一个传统组成部分是他们轮流提供的一盘点心。图 17-1显示了正在进行的迭代计划会议。

Teams new to agile development often need a lot of time for their iteration planning sessions. Iteration planning often took a whole day when Lisa’s team first started out. Now they are done in two or three hours, which includes time for the retrospective. Lisa’s team uses a projector to display user acceptance test cases and conditions of satisfaction from their wiki so that everyone on the team can see them. They also project their online story board tool, where they write the task cards. Another traditional component of their planning meetings is a plate of treats that they take turns providing. Figure 17-1 shows an iteration planning meeting in progress.

图 17-1迭代计划会议。经 Mike Thomas 许可使用。版权所有 2008。

Figure 17-1 Iteration planning meeting. Used with permission of Mike Thomas. Copyright 2008.

图像

学习细节

Learning the Details

理想情况下,产品负责人和/或其他客户团队成员参与迭代计划、回答问题并提供描述每个故事需求的示例。如果业务方面没有人可以参加,与客户密切合作的团队成员,如分析师和测试人员,可以作为代理人。他们代表客户解释细节并做出决定,或者记下问题以便快速得到答复。如果你的团队在迭代之前与客户一起回顾了故事,你可能认为在迭代计划会议期间你不需要他们在场。但是,我们建议您提供它们以防万一您有其他问题。

Ideally, the product owner and/or other customer team members participate in the iteration planning, answering questions and providing examples describing requirements of each story. If nobody from the business side can attend, team members who work closely with the customers, such as analysts and testers, can serve as proxies. They explain details and make decisions on behalf of the customers, or take note of questions to get answered quickly. If your team went over stories with the customers in advance of the iteration, you may think you don’t need them on hand during the iteration planning session. However, we suggest that they be available just in case you do have extra questions.

正如我们在整本书中所强调的那样,使用示例来帮助团队理解每个故事,并将这些示例转化为驱动编码的测试。按优先顺序处理故事。如果您之前没有与客户讨论过故事,则产品负责人或代表客户团队的其他人首先阅读要计划的每个故事。他们解释故事的目的、它将传递的价值,并举例说明如何使用它。这可能涉及传递示例或在白板上书写。UI 和报告故事可能已经有团队可以研究的线框或模型。

As we’ve emphasized throughout the book, use examples to help the team understand each story, and turn these examples into tests that drive coding. Address stories in priority order. If you haven’t previously gone over stories with the customers, the product owner or other person representing the customer team first reads each story to be planned. They explain the purpose of the story, the value it will deliver, and give examples of how it will be used. This might involve passing around examples or writing on a whiteboard. UI and report stories may already have wire frames or mock-ups that the team can study.

对某些团队有帮助的做法是在迭代计划期间为每个故事一起编写用户验收测试。他们与产品负责人和可能的其他利益相关者一起编写高级测试,当通过时,将表明故事已经完成。这也可以作为迭代“准备工作”的一部分在迭代计划之前不久完成。

A practice that helps some teams is to write user acceptance tests for each story together, during the iteration planning. Along with the product owner and possibly other stakeholders, they write high-level tests that, when passing, will show that the story is done. This could also be done shortly in advance of iteration planning as part of the iteration “prep work.”

故事的大小应该不超过几天就能完成。当我们定期测试小故事时,我们不会立即将它们全部完成并在迭代结束时堆积起来等待测试。如果一个故事已经通过了发布计划和预迭代讨论并且仍然太大,那么这是将它分解成更小的部分的最后机会。即使是一个小故事也可能很复杂。团队可能会通过练习来识别功能的薄片或关键路径。使用示例来指导您,找到最基本的用户场景。

Stories should be sized so they’ll take no more than a few days to complete. When we get small stories to test on a regular basis, we do not have them all finished at once and stacked up at the end of the iteration waiting to be tested. If a story has made it past release planning and pre-iteration discussions and is still too large, this is the final chance to break it up into smaller pieces. Even a small story can be complex. The team may go through an exercise to identify the thin slices or critical path through the functionality. Use examples to guide you, and find the most basic user scenarios.

敏捷测试人员以及其他团队成员对“范围蔓延”保持警惕。当一个故事似乎在四面八方发展时,不要害怕举起红旗。Lisa 的团队有意识地努力指出“bling”或“nice to have”组件,这些组件不是故事功能的核心。这些可以推迟到最后,或推迟,以防故事完成的时间比计划的要长。

Agile testers, along with other team members, are alert to “scope creep.” Don’t be afraid to raise a red flag when a story seems to be growing in all directions. Lisa’s team makes a conscious effort to point out “bling,” or “nice to have” components, which aren’t central to the story’s functionality. Those can be put off until last, or postponed, in case the story takes longer than planned to finish.

考虑所有观点

Considering All Viewpoints

作为测试人员,您将尝试将每个故事置于更大系统的上下文中,并评估对其他领域的意外影响的可能性。正如你在发布计划会议上所做的那样,让自己处于不同的心态用户、业务利益相关者、程序员、技术作家以及参与创建和使用该功能的每个人。现在您在详细级别上工作。

As a tester, you’ll try to put each story into the context of the larger system and assess the potential of unanticipated impacts on other areas. As you did in the release planning meeting, put yourself in the different mind-sets of user, business stakeholder, programmer, technical writer, and everyone involved in creating and using the functionality. Now you’re working at a detailed level.

在发布计划章节中,我们使用了这个示例故事:

In the release planning chapter, we used this example story:

作为客户,我想知道根据我选择的送货速度,我的订单需要支付多少运费,所以我可以根据需要进行更改。

As a customer, I want to know how much my order will cost to ship based on the shipping speed I choose, so I can change if I want to.

我们决定采取“薄片”并改变这个故事,假设只有一种运输速度。其他运输速度将在以后的故事。对于这个故事,我们需要根据物品重量和目的地计算运费,我们决定使用 BigExpressShipping 的 API 进行计算。我们的故事现在如图 17-2所示。

We decided to take a “thin slice” and change this story to assume there is only one shipping speed. The other shipping speeds will be later stories. For this story, we need to calculate shipping cost based on item weight and destination, and we decided to use BigExpressShipping’s API for the calculation. Our story is now as shown in Figure 17-2.

图 17-2 5 天交货的 Story 运输速度

Figure 17-2 Story shipping speed for 5-day delivery

图像

团队开始讨论这个故事。

The team starts discussing the story.

测试员: “这个故事是否适用于网站上所有可用的项目?是否有任何物品太重或不符合 5 天送达的条件?

Tester: “Does this story apply for all items available on the site? Are any items too heavy or otherwise disqualified for 5-day delivery?

产品负责人: “我们所有的产品都有 5 天的停产期。隔夜和 2 天的重量不得超过 25 磅。”

Product Owner: “5-day ground is available for all our items. It’s the overnight and 2-day that are restricted to less than 25 lbs.”

测试员: “从业务角度来看,这里的目标是什么?是否可以轻松计算加快结账的成本?你希望鼓励他们检查其他运输方式——那些更有利可图吗?”

Tester: “What’s the goal here, from the business perspective? Making it easy to figure the cost to speed up the checkout? Are you hoping to encourage them to check the other shipping methods—are those more profitable?”

产品负责人: “易用性是我们的主要目标,我们希望结账过程快速,我们希望用户轻松确定订单的总成本,这样他们就不会害怕完成购买。”

Product Owner: “Ease of use is our main goal, we want the checkout process to be quick, and we want the user to easily determine the total cost of the order so they won’t be afraid to complete the purchase.”

程序员: “只要用户输入送货地址,我们就可以默认显示 5 天的送货费用。当我们为其他运输选项制作故事时,我们可以放置按钮以快速弹出这些成本。”

Programmer: “We could have the 5-day shipping cost display as a default as soon as the user enters the shipping address. When we do the stories for the other shipping options, we can put buttons to pop up those costs quickly.”

产品负责人: “这就是我们想要的,预先计算成本。我们将把我们的网站宣传为对客户最友好的网站。”

Product Owner: “That’s what we want, get the costs up front. We’re going to market our site as the most customer friendly.”

测试员: “有什么方法可以让用户搞砸吗?他们会在这个页面上做什么?”

Tester: “Is there any way the user can screw up? What will they do on this page?”

产品负责人: “当我们添加其他运输选项时,他们可以选择更改他们的运输选项。但就目前而言,它真的很简单。我们已经进行了验证,以确保他们的邮政编码与他们为送货地址输入的城市相匹配。”

Product Owner: “When we add the other shipping options, they can opt to change their shipping option. But for now, it’s really straightforward. We already have validation to make sure their postal code matches the city they enter for the shipping address.”

测试员: “如果他们意识到他们弄乱了送货地址怎么办?也许他们不小心给了账单地址。他们怎么能回来更改送货地址?”

Tester: “What if they realize they messed up their shipping address? Maybe they accidentally gave the billing address. How can they get back to change the shipping address?”

程序员: “我们将放置按钮来编辑账单地址和送货地址,这样用户更容易更正错误。我们将在显示运费的页面上显示这两个地址。当我们添加多个送货地址选项时,我们可以稍后扩展它。”

Programmer: “We’ll put buttons to edit billing and shipping addresses, so it will be very easy for the user to correct errors. We’ll show both addresses on this page where the shipping cost displays. We can extend this later when we add the multiple shipping addresses option.”

测试人员: “这将使 UI 易于使用。我知道当我在网上购物时,在订单确认之前无法看到运费让我很烦恼。如果运费贵得离谱而且我不想继续,那我已经浪费了时间。我们希望确保用户不会卡在结账过程中,感到沮丧,然后放弃。因此,他们将看到的下一页是订单确认页面。当用户到达该页面时,运费是否有可能会有所不同?”

Tester: “That would make the UI easy to use. I know when I shop online, it bugs me to not be able to see the shipping cost until the order confirmation. If the shipping is ridiculously expensive and I don’t want to continue, I’ve already wasted time. We want to make sure users can’t get stuck in the checkout process, get frustrated, and just give up. So, the next page they’ll see is the order confirmation page. Is there any chance the shipping cost could be different when the user gets to that page?”

程序员: “不,只要相同的商品仍在购物车中,为我们提供估算成本的 API 就应该始终与实际成本相匹配。”

Programmer: “No, the API that gives us the estimated cost should always match the actual cost, as long as the same items are still in the shopping cart.”

产品负责人: “如果他们更改数量或删除任何项目,我们需要确保立即更改运费以反映这一点。”

Product Owner: “If they change quantities or delete any items, we need to make sure the shipping cost is immediately changed to reflect that.”

正如您在谈话中看到的那样,很多澄清都浮出水面。现在团队中的每个人都对这个故事有了共同的理解。谈论故事的各个方面很重要。以小组形式编写用户验收测试是确保开发团队了解客户需求的好方法。让我们继续监控这个对话。

As you can see by the conversation, a lot of clarification came to light. Everyone on the team now has a common understanding of the story. It’s important to talk about all aspects of the story. Writing user acceptance tests as a group is a good way to make sure the development team understands the customer requirements. Let’s continue monitoring this conversation.

测试人员: “让我们编写一些快速测试以确保我们做对了。”

Tester: “Let’s just write up some quick tests to make sure we get it right.”

客户: “好的,这个例子怎么样?

Customer: “OK, how’s this example?

我可以选择两件有 5 天送货选项的商品,并立即查看我的费用。

I can select two items with a 5-day shipping option and see my costs immediately.

测试员: “很好的开始,但到那时我们还不知道将它运送到哪里。更通用的测试怎么样:

Tester: “Great start, but we won’t know where to ship it to at that point. How about a more generic test like:

确认用户输入送货地址后,默认显示 5 天送货费用。

Verify the 5-day shipping cost displays as the default as soon as the user enters a shipping address.

顾客: “这对我有用。”

Customer: “That works for me.”

基于不同的观点提出问题将有助于澄清故事并让团队做得更好。

Asking questions based on different viewpoints will help to clarify the story and allow the team to do a better job.

编写任务卡

Writing Task Cards

当您的团队对故事有了很好的理解后,您就可以开始编写和估算任务卡了。因为敏捷开发通过测试驱动编码,所以我们同时编写测试和开发任务卡。

When your team has a good understanding of a story, you can start writing and estimating task cards. Because agile development drives coding with tests, we write both testing and development task cards at the same time.

如果你做了任何预先计划,你可能已经写好了一些任务卡。如果没有,在迭代计划会议期间写下它们。谁写任务卡并不重要,但团队中的每个人都应该审阅它们并有机会提供他们的意见。我们认识到,在我们开始编码时可能会添加任务,但是在会议期间识别大部分任务并对其进行估算可以让团队很好地了解所涉及的内容。

If you have done any pre-planning, you may have some task cards already written out. If not, write them during the iteration planning meeting. It doesn’t matter who writes the task cards, but everyone on the team should review them and get a chance to give their input. We recognize that tasks may be added as we begin coding, but recognizing most of the tasks and estimating them during the meeting gives the team a good sense of what is involved.

丽莎的故事

Lisa’s Story

当我们的团队准备开始编写任务卡时,程序员通常会拿出编码任务卡。测试人员同时编写测试任务卡。

When our team is ready to start writing task cards, programmers usually come up with the coding task cards. The testers write testing task cards at the same time.

我通常从一张卡片开始写高级测试用例。我问程序员这个故事是否可以在 GUI 后面进行测试,并相应地编写测试任务卡。这通常意味着一张用于“编写 FitNesse 测试用例”的测试卡和一张用于“编写 FitNesse 夹具”的开发人员任务卡,除非夹具已经存在。有时在单元测试中可以更容易地覆盖所有 GUI 背后的测试,所以询问是否是这种情况总是好的。

I usually start with a card to write high-level test cases. I ask the programmers whether the story can be tested behind the GUI, and write testing task cards accordingly. This usually means a test card to “Write FitNesse test cases” and a developer task card to “Write FitNesse fixture,” unless the fixtures already exist. Sometimes all of the behind-GUI tests can be covered more easily in unit tests, so it is always good to ask whether this is the case.

我们将团队在迭代期间需要记住的任何内容都放在任务卡上。“向 Anne 显示 UI”或“向 Joe 发送测试文件”与所有其他任务一起出现在故事​​板上。

We put anything the team needs to remember during the iteration on a task card. “Show UI to Anne” or “Send test files to Joe” go up on the story board along with all of the other tasks.

我边走边估算测试任务卡,并向团队征求关于卡片和估算的反馈。有时我们分成几个小组,每个小组拿一些故事,给他们写任务卡。我们总是一起审查所有的卡片,以及估计的时间。如果与测试时间相比开发时间相对较短,或者相反,这会引发讨论。我们就我们是否认为故事的所有方面都被任务卡覆盖达成了共识。如果还有一些未知数,我们只是推迟写任务卡,直到我们掌握了信息。

I estimate the testing task cards as I go, and ask the team for feedback on the cards and the estimates. Sometimes we divide into groups, and each group takes some stories and writes task cards for them. We always review all cards together, along with the estimated time. If development time is relatively low compared to testing time, or vice versa, that provokes a discussion. We reach a consensus as to whether we think all aspects of the story have been covered with task cards. If there are still some unknowns, we simply postpone writing the task cards until we have the information.

测试卡和开发卡都放在故事板上的“待办事项”栏中。团队中的任何人都可以注册任何卡。一些测试任务卡在编码卡开始移动之前移动到“正在进行的工作”或“已完成”列,以便程序员有一些测试来指导他们的编码。随着编码任务卡移至“已完成”列,用于测试“已完成”功能的卡片将移至“进行中”。

The testing and development cards all go on the story board in the “to do” column. Anyone on the team can sign up for any card. Some testing task cards move to the “work in progress” or “done” column before coding cards start to move, so that programmers have some tests to guide their coding. As coding task cards are moved to the “done” column, the cards for testing the “done” functionality are moved into “work in progress.”

——丽莎

—Lisa

Janet 使用的方法与此类似,但程序员的编码卡停留在“待测试”列中,直到测试任务完成。两张卡片同时移动到“完成”列。

Janet uses an approach similar to this, but the programmer’s coding card stays in the “To Test” column until the testing task has been completed. Both cards move at the same time to the “Done” column.

Story PA-5(图 17-2)的三张测试卡显示了基于重量和目的地的 5 天送达的运费,Lisa 的团队可能会写下这些:

Three test cards for Story PA-5 (Figure 17-2), displaying the shipping cost for 5-day delivery based on weight and destination, that Lisa’s team might write are:

图像编写 FitNesse 测试以根据重量和目的地计算 5 天的运输成本。

Write FitNesse tests for calculating 5-day ship cost based on weight and destination.

图像编写用于显示 5 天运送成本的 WebTest 测试。

Write WebTest tests for displaying the 5-day ship cost.

图像手动测试显示 5 天交货运费。

Manually test displaying the 5-day delivery ship cost.

有些团队更喜欢直接在开发任务卡上写测试任务。这是一个简单的解决方案,因为在测试完成之前任务显然没有“完成”。您正在尝试避免最后完成测试的“迷你瀑布”方法,而程序员觉得她已经完成了,因为她“将故事发送给了 QA”。查看哪种方法最适合您的团队。

Some teams prefer to write testing tasks directly on the development task cards. It’s a simple solution, because the task is obviously not “done” until the testing is finished. You’re trying to avoid a “mini-waterfall” approach where testing is done last, and the programmer feels she is done because she “sent the story to QA.” See what approach works best for your team.

如果故事大量涉及外部各方或共享资源,请编写任务卡以确保不会忘记这些任务,并使估算足够大以允许团队无法控制的依赖性和事件。我们处理运输成本故事的假设团队必须使用托运人的成本计算 API。

If the story heavily involves outside parties or shared resources, write task cards to make sure those tasks aren’t forgotten, and make the estimates generous enough to allow for dependencies and events beyond the team’s control. Our hypothetical team working on the shipping cost story has to work with the shipper’s cost calculation API.

测试员: “有谁知道我们在 BigExpressShipping 与谁合作以获得他们的 API 规范?我们传递给他们什么,只是重量和邮政编码?我们已经可以进行测试了吗?”

Tester: “Does anyone know who we work with at BigExpressShipping to get specs on their API? What do we pass to them, just the weight and postal code? Do we already have access for testing this?”

Scrum Master: “BigExpressShipping 的 Joe 是我们的联系人,他已经发送了这份指定输入和输出格式的文档。他们仍然需要授权从我们的测试系统访问,但这应该在几天内完成。”

Scrum Master: “Joe at BigExpressShipping is our contact, and he’s already sent this document specifying input and output format. They still need to authorize access from our test system, but that should be done in a couple of days.”

测试人员: “哦,很好,我们需要这些信息来编写测试用例。我们将编写一张测试卡来验证我们是否可以访问他们的 API 并收回运费。但是我们怎么知道成本真的是正确的呢?”

Tester: “Oh good, we need that information to write test cases. We’ll write a test card just to verify that we can access their API and get a shipping cost back. But how do we know the cost is really correct?”

Scrum Master: “Joe 为我们提供了一些重量和邮政编码以及预期成本的测试用例,因此我们可以发送这些输入并检查正确的输出。我们还有此电子表格显示一些不同邮政编码的费率。”

Scrum Master: “Joe has provided us with some test cases for weight and postal code and expected cost, so we can send those inputs and check for the correct output. We also have this spreadsheet showing rates for some different postal codes.”

测试人员: “我们应该留出大量时间来确保我们正确访问他们的 API。我打算对这张卡进行高估,以验证使用API​​ 进行测试。也许 API 接口的开发者卡也应该有一个相当保守的估计。”

Tester: “We should allow lots of time for just making sure we’re accessing their API correctly. I’m going to put a high estimate on this card to verify using the API for testing. Maybe the developer card for the interface to the API should have a pretty conservative estimate, too.”

在编写程序员任务卡时,确保编码任务估计包括编写单元测试和程序员进行所有必要测试的时间。用于“端到端”测试的卡片有助于确保从事不同、独立任务的程序员验证所有部分是否协同工作。测试人员应该帮助确保写下所有必要的卡片并且他们有合理的估计。您不希望事后估计,但如果测试估计是编码估计的两倍,则可能值得一谈。

When writing programmer task cards, make sure that coding task estimates include time for writing unit tests and for all necessary testing by programmers. A card for “end-to-end” testing helps make sure that programmers working on different, independent tasks verify that all of the pieces work together. Testers should help make sure all necessary cards are written and that they have reasonable estimates. You don’t want second-guess estimates, but if the testing estimates are twice as high as the coding estimates, it might be worth talking about.

一些团队将测试任务限制在一天或更少的工作时间内,并且懒得在卡片上写下估计的时间。如果一天的工作后任务卡仍然存在,团队会讨论发生这种情况的原因并写新卡以继续前进。这可能会减少开销和记录保存,但如果您将任务输入电子系统,则可能不会。做对您的团队有意义的事情。

Some teams keep testing tasks to a day’s work or less and don’t bother to write estimated hours on the card. If a task card is still around after a day’s work, the team talks about why that happened and writes new cards to go forward. This might cut down on overhead and record-keeping, but if you are entering tasks into your electronic system, it may not. Do what makes sense for your team.

估计错误修复的时间也总是很棘手。如果将现有缺陷作为故事引入,那将非常简单。但是作为迭代的一部分发现的错误呢?

Estimating time for bug fixing is always tricky as well. If existing defects are pulled in as stories, it is pretty simple. But what about the bugs that are found as part of the iteration?

珍妮特的故事

Janet’s Story

对于新的敏捷团队,我发现他们似乎总是把时间花在错误上,而这些错误并没有作为他们对故事的估计的一部分进行分配。随着时间的推移,程序员会了解到他们通常会花多少时间来修复故事中的错误,并且可以为此增加半天或几个小时的任务。重新测试错误修复也会增加测试人员的估计时间。

With new agile teams, I found that they always seem to end up with time spent on bugs that wasn’t allotted as part of their estimates for the stories. Over time, programmers learn how much time they typically spend fixing bugs from a story, and can just add half a day or a couple hours to their tasks for that purpose. Retesting bug fixes adds time to tester’s estimates as well.

在团队成员解决这个问题之前,最好分别跟踪修复和测试错误所花费的时间。我目前的团队在 XPlanner 中添加了一个故事,其中包含修复和测试那些没有立即发现的错误的任务。我们正在跟踪时间,以便我们可以更好地估计未来的情况。

Until the team members get a handle on this, it may be appropriate to track the time spent on fixing and testing bugs separately. My current team adds a story in XPlanner with tasks for fixing and testing those bugs that didn’t get caught immediately. We are tracking the time so we can better estimate down the road.

——珍妮特

—Janet

无论您的团队选择估算在迭代期间修复缺陷所花费的时间,无论是包含在故事估算中还是单独跟踪,都要确保始终如一地完成。

However your team chooses to estimate time spent for fixing defects during the iteration, whether it is included in the story estimate or tracked separately, make sure it is done consistently.

估计测试任务时要考虑的另一个项目是测试数据。迭代的开始几乎来不及考虑您需要测试的数据。正如我们在第 15 章发布或主题规划中的测试人员活动”中提到的,在发布规划期间考虑测试数据,并要求客户帮助识别和获取它。在为下一次迭代做准备时,一定要考虑一下。当迭代开始时,必须创建或获取任何缺少的测试数据,所以不要忘记在估算中考虑到这一点。

Another item to consider when estimating testing tasks is test data. The beginning of an iteration is almost too late to think about the data you need to test with. As we mentioned in Chapter 15, “Tester Activities in Release or Theme Planning,” think about test data during release planning, and ask the customers to help identify and obtain it. Certainly think about it as you prep for the next iteration. When the iteration starts, whatever test data is missing must be created or obtained, so don’t forget to allow for this in estimates.

丽莎的故事

Lisa’s Story

我们研究了一个与退休计划参与者的季度账户报表相关的主题。我们正在修改一个月度作业,该作业在指定日期拍摄每个参与者帐户的“快照”。快照依赖于生产数据库中的海量数据,包括数以千计的日常事务。我们提前计划好了。

We worked on a theme related to quarterly account statements for participants in retirement plans. We were modifying a monthly job that takes a “snapshot” of each participant’s account on the specified date. The snapshot relies on a huge amount of data in the production database, including thousands of daily transactions. We planned ahead.

对于第一次迭代,我们在主题中做了一些故事,我们知道我们只能使用我们在测试数据库中有数据的一些个人退休计划来测试几个案例。我们还知道我们需要进行更大规模的测试,数据库中包含所有退休计划以及至少一整月的数据。我们写了一张任务卡来复制足够的生产数据来产生一个每月的“快照”,并确保数据被清理以保护隐私。

For the first iteration, we did a few stories in the theme knowing we could only test a few cases using some individual retirement plans for which we had data in the test database. We also knew we needed a larger-scale test, with all of the retirement plans in the database and at least an entire month’s worth of data. We wrote a task card to copy enough production data to produce one monthly “snapshot” and made sure the data was scrubbed to protect privacy.

然后我们计划在下一次迭代中进行全面测试。这些数据使测试人员能够发现早期只有部分数据可用时无法检测到的问题。这是完成大部分编码的“刚好足够”的数据和及时可用的全部数据以验证完整功能的很好的平衡。由于团队提前计划,因此在关键版本发布之前及时修复了错误。

Then we planned the full-blown test in the next iteration. This data enabled the testers to find problems that were undetectable earlier when only partial data was available. It was a nice balance of “just enough” data to do most of the coding and the full amount available in time to verify the complete functionality. Because the team planned ahead, the bugs were fixed in time for the critical release.

——丽莎

—Lisa

决定工作量

Deciding on Workload

作为技术团队,我们控制自己的工作量。当我们为每个故事编写任务并将它们发布到我们的(真实的或虚拟的)故事板上时,我们将估计的时间相加或目视检查卡片的数量。我们可以承担多少工作?在 XP 中,我们不能超过上一次迭代中完成的故事点数。在 Scrum 中,我们根据我们认为需要完成的实际时间来承诺一组故事。

We, as the technical team, control our own workload. As we write tasks for each story and post them on our (real or virtual) story board, we add up the estimated hours or visually check the number of cards. How much work can we take on? In XP, we can’t exceed the number of story points we completed in the last iteration. In Scrum, we commit to a set of stories based on the actual time we think we need to complete them.

Lisa 当前的团队在他们的敏捷过程中拥有多年的经验,并且发现他们有时会浪费时间为他们在迭代期间可能没有时间完成的故事编写任务卡。他们从足够多的故事开始,让每个人都忙起来。随着人们开始腾出时间,他们会收集更多故事并计划相关任务。他们可能已经准备好一些故事,以便在完成最初的故事后立即引入。这听起来很容易,但要做到这一点很难,除非您学到了足够多的知识,对故事的规模和团队速度更有信心,并且知道您的团队在给定的时间和特定情况下可以做什么和不能做什么。

Lisa’s current team has several years of experience in their agile process and finds they sometimes waste time writing task cards for stories they may not have time to do during the iteration. They start with enough stories to keep everyone busy. As people start to free up, they pull in more stories and plan the related tasks. They might have some stories ready “on deck” to bring in as soon as they finish the initial ones. This sounds easy, but it is difficult to do until you’ve learned enough to be more confident about story sizes and team velocity, and know what your team can and cannot do in a given amount of time and in specific circumstances.

作为测试人员,您的工作是确保为测试分配足够的时间,并提醒团队测试和质量是整个团队的责任。当团队决定他们可以在迭代中交付多少故事时,问题不是“我们可以完成多少编码?” 但是“我们可以完成多少编码和测试?” 有时故事很容易编码,但测试会非常耗时。作为测试人员,重要的是您只接受尽可能多的故事进入迭代,可​​以测试。

Your job as tester is to make sure enough time is allocated to testing, and to remind the team that testing and quality are the responsibility of the whole team. When the team decides how many stories they can deliver in the iteration, the question isn’t “How much coding can we finish?” but “How much coding and testing can we complete?” There will be times when a story is easy to code but the testing will be very time consuming. As a tester, it is important that you only accept as many stories into the iteration as can be tested.

如果您必须提交,请谨慎提交。引入另一个故事总是比不得不放弃一个更好。如果您有难以估计的高风险故事,或者某些任务未知或需要更多研究,请为一两个额外的故事写下任务卡,并在场边准备好它们以进行中期迭代。

If you have to commit, commit conservatively. It’s always better to bring in another story than to have to drop one. If you have high-risk stories that are hard to estimate, or some tasks are unknown or need more research, write task cards for an extra story or two and have them ready on the sidelines to bring in mid-iteration.

作为一个团队,我们总是会尽力而为。我们需要记住,只有经过测试,故事才能完成,因此请相应地进行计划。

As a team, we’re always going to do our best. We need to remember that no story is done until it’s tested, so plan accordingly.

可测试的故事

Testable Stories

当你在看故事时,程序员开始考虑实现,总是想着如何测试它们。一个例子对“测试可测试性”大有帮助。它会对我的测试产生什么影响?第 III 部分,“敏捷测试象限”,给出了很多关于如何设计应用程序以实现有效测试。这是您在编码开始之前考虑故事的可测试性的最后机会。

When you are looking at stories, and the programmers start to think about implementation, always think how you can test them. An example goes a long way toward “testing the testability.” What impact will it have on my testing? Part III, “The Agile Testing Quadrants,” gives a lot of examples of how to design the application to enable effective testing. This is your last opportunity to think about testability of a story before coding begins.

珍妮特的故事

Janet’s Story

与我合作的一个团队告诉我他们在上一个版本中遇到的问题。该团队正在重写多步骤流程的第一步。他们没有预料到的是,当新步骤的开发开始时,剩下的流程就中断了。在整个第一步完成之前,无法对该迭代中的任何其他更改进行测试。

One team I worked with told me about issues they had in the previous release. The team was rewriting the first step of a multistep process. What they didn’t anticipate was that when the development on the new step started, the rest of the process broke. No testing could be done on any other changes in that iteration until the whole first step was finished.

在计划故事时没有考虑可测试性。在下一个版本中,当他们决定重写第二步时,他们从之前的错误中吸取了教训。程序员在页面上创建了一个额外的按钮,允许测试人员调用新页面(不断变化)或旧页面以允许他们测试其他故事。

Testability had not been considered when planning the story. In the next release, when they decided to rewrite the second step, they learned from their previous mistake. The programmers created an extra button on the page that allowed the testers to either call the new page (in flux) or the old page to allow them test other stories.

记得问,“我们如何测试这个?” 如果这对你来说不是很明显。

Remember to ask, “How can we test this?” if it is not obvious to you.

——珍妮特

—Janet

在迭代计划期间,考虑需要测试哪种变体。这可能会引发其他问题。

During iteration planning, think about what kind of variations you will need to test. That may drive other questions.

珍妮特的故事

Janet’s Story

在我参加的一次迭代计划会议上,程序员们开始谈论实现并在白板上画图来展示他们的想法。

During one iteration planning meeting that I was in, the programmers started talking about implementation and drawing pictures on the whiteboard to show what they were thinking.

我想了想,问道:“能不能简单点?” 用于测试您提议的实施的排列和组合将使测试变得可怕。”

I thought about it for a bit and asked the question, “Can it be done more simply? The permutations and combinations for testing your proposed implementation will make testing horrendous.”

程序员考虑了几分钟并提出了一个替代方案,该方案不仅可以满足客户的需求,而且更简单、更易于测试。这对每个人来说都是双赢的组合。

The programmers thought about it for a couple of minutes and suggested an alternative that not only met the customer’s needs, but was simpler and easier to test. It was a win-win combination for everyone.

——珍妮特

—Janet

当可测试性成为问题时,将其作为团队的问题来解决。通过编写测试任务卡开始计划的团队可能在这方面有优势,因为当他们考虑他们的测试任务时,他们会问如何测试这个故事。是否可以在 GUI 后面测试任何功能?是否可以在单元级别进行面向业务的测试?每个敏捷团队都应该首先考虑测试。当您的团队为故事编写开发人员任务卡时,请考虑如何测试故事以及如何对其进行自动化测试。如果程序员还没有编写 TDD 代码或自动化单元测试的习惯,请尝试为每个故事写一张“XUnit”任务卡。为任何需要的测试自动化装置编写编程任务卡。考虑有助于测试的应用程序更改,例如运行时属性和 API。

When testability is an issue, make it the team’s problem to solve. Teams that start their planning by writing test task cards probably have an advantage here, because as they think about their testing tasks, they’ll ask how the story can be tested. Can any functionality be tested behind the GUI? Is it possible to do the business-facing tests at the unit level? Every agile team should be thinking test-first. As your team writes developer task cards for a story, think about how to test the story and how to automate testing for it. If the programmers aren’t yet in the habit of coding TDD or automating unit tests, try writing a “XUnit” task card for each story. Write programming task cards for any test automation fixtures that will be needed. Think about application changes that could help with testing, such as runtime properties and APIs.

丽莎的故事

Lisa’s Story

我处理的应用程序有很多时间和日期相关的活动。程序员向 Web 应用程序添加了运行时服务器属性以设置服务器日期。我可以指定一个日期和时间覆盖,当服务器启动时,它会相应地运行。这允许通过简单的覆盖启动每月或每季度的流程。此属性有助于测试各种故事。

The application that I work on has many time- and date-dependent activities. The programmers added a runtime server property to the web application to set the server date. I can specify a date and time override, and when the server starts up, it behaves accordingly. This allows kicking off monthly or quarterly processes with a simple override. This property has helped in testing a wide variety of stories.

Markus Gärtner [2008] 告诉我们他的团队有一个类似的属性,一个以“提前天数”计算的“DATE_OFFSET”。但是,这仅由业务逻辑所在的应用程序的 Java 组件使用。C 和 C++ 中的后端系统不使用日期偏移量,这导致了问题。

Markus Gärtner [2008] told us his team has a similar property, a “DATE_OFFSET” counted in “days to advance.” However, this was only used by the Java components of the application where the business logic lives. The back-end systems in C and C++ don’t use the date offset, which caused a problem.

——丽莎

—Lisa

如果您因为其他团队正在开发系统的某些部分而遇到类似问题,请编写任务卡与其他团队讨论该问题并提出协调的解决方案。如果无法与其他团队合作,请安排时间集思广益,想出另一个解决方案。至少,要注意这些限制,并相应地调整测试估计并管理相关风险。

If you have similar issues because other teams are developing parts of the system, write a task card to discuss the problem with the other team and come up with a coordinated solution. If working with the other team isn’t an option, budget time to brainstorm another solution. At the very least, be mindful of the limitations, and adjust testing estimates accordingly and manage the associated risk.

丽莎的故事

Lisa’s Story

我们启动了一个项目来替换我们公司的交互式语音应答 (IVR) 系统,该系统允许退休计划参与者通过电话获取账户信息和管理账户。我们与另一家公司签约用 Java 编写系统,目的是让我们的团队在一段时间后维护它。

We started a project to replace our company’s interactive voice response (IVR) system, which allows retirement plan participants to obtain account information and manage accounts by phone. We contracted with another company to write the system in Java, with the intention that our team would maintain it after a certain time period.

我们花了一些时间集思广益,讨论需要进行哪些测试以及如何进行。据推测,承包商会测试诸如文本到语音功能之类的东西,但我们必须提供存储过程以从数据库中检索适当的数据。

We spent some time brainstorming what testing would be needed and how to do it. Presumably, the contractor would test things like the text-to-speech functionality, but we had to supply stored procedures to retrieve appropriate data from the database.

我们的第一步是与承包商协商,在迭代的基础上交付小块功能,这样它们就可以随着项目的进展进行测试,并且工作将在合同的整个生命周期内平均分配。我们决定使用 FitNesse fixture 测试存储过程,并探索各种选项。我们决定使用 PL/SQL 来访问存储过程。一名程序员的任务是加快 PL/SQL 的速度,以解决测试自动化问题。

Our first step was to negotiate with the contractor to deliver small chunks of features on an iteration basis, so they could be tested as the project progressed and the work would be spread out evenly over the life of the contract. We decided to test the stored procedures using FitNesse fixtures, and explored the options. We settled on PL/SQL to access the stored procedures. A programmer was tasked with getting up to speed on PL/SQL to tackle the test automation.

该团队旨在采用循序渐进的方法。通过在开始时为任务分配大量时间,我们允许涉及陡峭的学习曲线。

The team aimed for a step-by-step approach. By allocating plenty of time for tasks at the start, we allowed for the steep learning curves involved.

有趣的是,承包商为第一次迭代交付了初始构建,但无法为接下来的几次迭代交付代码增量。我们最终取消了合同并推迟了项目,直到我们找到更好的解决方案。通过强迫承包商以增量方式工作,我们立即发现它无法交付。如果我们让他们花六个月时间来编写整个应用程序会怎么样?它可能不会结束。我们将所学知识用于研究更好的方法。

Interestingly, the contractor delivered an initial build for the first iteration but was not able to deliver the increments of code for the next few iterations. We ended up canceling the contract and postponing the project until we could find a better solution. By forcing the contractor to work in increments, we discovered right away that it couldn’t deliver. What if we had let them take six months to write the whole application? It probably wouldn’t have ended well. We put what we learned to good use in researching a better approach.

——丽莎

—Lisa

当您开始对团队进行一些新事物时,例如新的模板框架或报告库,请记住将其作为风险包含在您的测试计划中。希望您的团队在选择新框架或工具之前考虑了可测试性,并选择了一个可以增强您的测试能力的框架或工具。对所有新事物(包括新领域)的测试任务估计要大方,因为有很多未知数。有时新的领域知识或新技术意味着陡峭的学习曲线。

When you’re embarking on something new to the team, such as a new templating framework or reporting library, remember to include it as a risk in your test plan. Hopefully, your team considered the testability before choosing a new framework or tool, and selected one that enhanced your ability to test. Be generous with your testing task estimates with everything new, including new domains, because there are lots of unknowns. Sometimes new domain knowledge or new technology means a steep learning curve.

与客户合作

Collaborate with Customers

与客户或客户代理(如功能分析师)密切合作是我们作为敏捷测试人员最重要的活动之一。当您开始迭代时,您的客户协作也将进入高潮。现在是执行第 8 章支持团队的面向业务的测试”中描述的所有那些好的活动的时候了。向客户询问示例,询问有关每个故事的功能和行为的开放式问题,围绕白板进行讨论,然后将这些示例转化为测试以驱动编码。

Working closely with customers, or customer proxies such as functional analysts, is one of our most important activities as agile testers. As you kick off the iteration, your customer collaboration will also kick into high gear. This is the time to do all those good activities described in Chapter 8, “Business-Facing Tests that Support the Team.” Ask the customers for examples, ask open-ended questions about each story’s functionality and behavior, have discussions around the whiteboard, and then turn those examples into tests to drive coding.

即使您的产品负责人和/或其他客户在迭代计划之前和期间解释了这些故事,有时在迭代开始时再简单地复习一遍它们也是有帮助的。以前可能不是每个人都听说过,客户可能了解更多。

Even if your product owner and/or other customers explained the stories before and during iteration planning, it’s sometimes helpful to go over them briefly one more time as the iteration starts. Not everyone may have heard it before, and the customer may have more information.

丽莎的故事

Lisa’s Story

我们在迭代的第一天开始编写高级验收测试。因为我们在迭代前一天与产品负责人一起检查所有故事,并作为一个团队为更复杂的故事编写用户验收测试,所以我们非常清楚需要什么。然而,编写更多测试用例的行为常常会带来新的问题。我们与产品负责人讨论了高级测试和任何问题,他们也一直在思考这些故事。

We start writing high-level acceptance tests the first day of the iteration. Because we go over all stories with the product owner the day before the iteration and write user acceptance tests as a team for the more complex stories, we have a pretty good idea of what’s needed. However, the act of writing more test cases often brings up new questions. We go over the high-level tests and any questions we have with the product owner, who has also been thinking more about the stories.

这方面的一个例子是一个故事,其中涉及一份货币分配文件,以计划从退休账户中提取资金的参与者。该文件被发送给合作伙伴,该合作伙伴使用该信息来削减参与者的支票。一些记录中的金额在合作伙伴的系统中没有正确核对,合作伙伴要求提供一个包含金额的新列以允许他们进行核对。

One example of this was a story that involved a file of monetary distributions to plan participants who withdraw money from their retirement accounts. This file is sent to a partner who uses the information to cut checks to the participant. The amounts in some of the records were not reconciling correctly in the partner’s system, and the partner asked for a new column with an amount to allow them to do a reconciliation.

在迭代计划会议之后,我们的产品负责人开始担心新专栏不是正确的解决方案,并在故事审查会议上提出了他的疑虑。他和一名测试人员进一步研究了这个问题,发现需要更改计算而不是添加新的金额。这实际上是一个更大的故事,但它解决了发行版的核心问题。团队讨论了更大的故事并编写了新的任务卡。花一点时间进一步讨论这个故事是值得的,因为最初的理解被证明是错误的。

After the iteration planning meeting, our product owner became concerned that the new column wasn’t the right solution and brought up his misgivings in the story review meeting. He and a tester studied the problem further and found that instead of adding a new amount, a calculation needed to be changed. This was actually a bigger story, but it addressed a core issue with the distributions. The team discussed the larger story and wrote new task cards. It was worth taking a little time to discuss the story further, because the initial understanding turned out to be wrong.

——丽莎

—Lisa

良好的沟通通常需要努力。如果您没有抓住足够的机会提出问题和审查测试用例,请继续并安排定期会议来这样做。如果没有太多要讨论的,会议会进行得很快。在会议中花时间进行有见地的讨论可以节省以后的编码和测试时间,因为您对需求更加确定。

Good communication usually takes work. If you’re not taking enough opportunities to ask questions and review test cases, go ahead and schedule regular meetings to do so. If there’s not much to discuss, the meetings will go quickly. Time in a meeting for an insightful discussion can save coding and testing time later, because you’re more certain of the requirements.

高级测试和示例

High-Level Tests and Examples

我们想要“大局”测试来帮助程序员在故事的正确方向上开始。像往常一样,我们建议从示例开始并将它们转化为测试。在编码开始之前,您必须进行试验,看看在验收测试级别有多少细节是合适的。Lisa 的团队发现,从示例中提取的高级测试是他们开始故事所需要的。

We want “big picture” tests to help the programmers get started in the right direction on a story. As usual, we recommend starting with examples and turning them into tests. You’ll have to experiment to see how much detail is appropriate at the acceptance test level before coding starts. Lisa’s team has found that high-level tests drawn from examples are what they need to kick off a story.

高级测试应该传达故事背后的主要目的。它们可能包括期望和不期望行为的示例。对于我们之前要求根据订单重量和目的地显示 5 天送达的运费的故事 PA-5(图 17-2),我们的高级测试可能包括:

High-level tests should convey the main purpose behind the story. They may include examples of both desired and undesired behavior. For our earlier Story PA-5 (Figure 17-2) that asks to show the shipping cost for 5-day delivery based on the order’s weight and destination, our high-level tests might include:

图像验证用户输入送货地址后 5 天的送货费用是否显示为默认值。

Verify that the 5-day shipping cost displays as the default as soon as the user enters a shipping address.

图像验证估计的运费是否与最终发票上的运费相符。

Verify that the estimated shipping cost matches the shipping cost on the final invoice.

图像 验证用户是否可以单击按钮来更改送货地址,完成后会显示更新的运费。

Verify that the user can click a button to change the shipping address, and when this is done, the updated shipping cost displays.

图像验证如果用户从购物车中删除项目或将项目添加到购物车,则显示更新的运输选项。

Verify that if the user deletes items from the cart or adds items to the cart, the updated shipping option is displayed.

编写高级测试时,不要局限于 wiki 页面上的文字。例如,如图 15-7所示的测试矩阵可能会工作得更好。有些人使用工作流绘图和图片以图形方式表达测试。Brian Marick [ 2007 ] 有一种绘制图形测试的技术,可以将其转换为 Ruby 测试脚本。模型驱动开发提供了另一种方式来表达故事的高级范围。用例是在“大局”级别表达所需行为的另一种可能途径。

Don’t confine yourself to words on a wiki page when you write high-level tests. For example, a test matrix such as the one shown in Figure 15-7 might work better. Some people express tests graphically, using workflow drawings and pictures. Brian Marick [2007] has a technique to draw graphical tests that can be turned into Ruby test scripts. Model-driven development provides another way to express high-level scope for a story. Use cases are another possible avenue for expressing desired behavior at the “big picture” level.

请参阅参考书目以获取有关图形测试和模型驱动开发的更多信息的链接。

See the bibliography for links to more information on graphical tests and model-driven development.

模型可以快速清晰地传达对 UI 或报告的要求。如果现有报告需要修改,请截取报告的屏幕截图并使用荧光笔、钢笔、铅笔或任何方便的工具。如果您想以电子方式捕获它,请尝试使用 Windows 画图程序或其他图形工具绘制更改并将其发布在描述报告要求的维基页面上。

Mock-ups can convey requirements for a UI or report quickly and clearly. If an existing report needs modifying, take a screenshot of the report and use highlighters, pen, pencil, or whatever tools are handy. If you want to capture it electronically, try the Windows Paint program or other graphical tool to draw the changes and post it on the wiki page that describes the report’s requirements.

请参阅第 16 章开始运行”中的 UI 更改示例模型。

See the sample mock-up of UI changes in Chapter 16, “Hit the Ground Running.”

分布式团队需要以电子方式提供的高级测试,而同一地点的团队可以通过白板上的绘图工作得很好,甚至可以让客户坐在他们身边并在他们编码时告诉他们需求。

Distributed teams need high-level tests available electronically, while co-located teams might work well from drawings on a whiteboard, or even from having the customer sit with them and tell them the requirements as they code.

请参阅第 9 章支持团队的面向业务的测试工具包”,了解有关收集和交流需求的工具的一些想法。

See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for some ideas on tools to gather and communicate requirements.

在开始迭代时,重要的是您要快速了解每个故事的基本要求,并以适用于整个团队的方式在上下文中表达它们。我们谈过的大多数敏捷团队都说他们最大的问题是要足够好地理解每个故事,以准确地提供客户想要的东西。他们可能会生成技术上没有错误但与客户所需功能不完全匹配的代码。或者,随着客户澄清需求,他们最终可能会在迭代期间对一个故事进行大量返工,结果没时间完成另一个故事。

What’s important as you begin the iteration is that you quickly learn the basic requirements for each story and express them in context in a way that works for the whole team. Most agile teams we’ve talked to say their biggest problem is to understand each story well enough to deliver exactly what the customer wanted. They might produce code that’s technically bug-free but doesn’t quite match the customer’s desired functionality. Or they may end up doing a lot of rework on one story during the iteration as the customer clarifies requirements, and run out of time to complete another story as a result.

花时间和精力尝试不同的方法,以适合您的领域和环境的方式捕获和表达高级测试。Janet 喜欢说需求是故事 + 对话 + 用户场景或支持图片(如果需要)+ 辅导测试或示例的组合。

Put time and effort into experimenting with different ways to capture and express the high-level tests in a way that fits your domain and environment. Janet likes to say that a requirement is a combination of the story + conversation + a user scenario or supporting picture if needed + a coaching test or example.

请参阅第 8 章,“支持团队的面向业务的测试”,了解更多关于构成需求的内容。

See Chapter 8, “Business-Facing Tests that Support the Team,” for more about what makes up a requirement.

与客户一起审查

Reviewing with Customers

在本章的前面,我们谈到了持续的客户协作的重要性。与客户一起审查高级测试是加强协作和加强沟通的好机会,尤其是对于新的敏捷团队而言。在您的团队习惯于不断讨论故事、需求和测试用例之后,您可能不需要坐下来仔细检查每个测试用例。

Earlier in this chapter we talked about the importance of constant customer collaboration. Reviewing high-level tests with customers is a good opportunity for enforced collaboration and enhanced communication, especially for a new agile team. After your team is in the habit of continually talking about stories, requirements, and test cases, you might not need to sit down and go over every test case.

如果您的团队签订合同开发软件,需求和测试用例可能是您必须提交的正式交付物。即使不是,以客户可以轻松阅读和理解的格式提供测试用例也是一个好主意。

If your team is contracting to develop software, requirements and test cases might be formal deliverables that you have to present. Even if they aren’t, it’s a good idea to provide the test cases in a format that the customers can easily read on their own and understand.

与程序员一起审查

Reviewing with Programmers

你可以拥有世界上所有的图表和 wiki 页面,但如果没有人看它们,它们将无济于事。直接沟通总是最好的。与程序员坐下来讨论高级测试和需求。一起检查白板图或纸质原型。图 17-4显示了测试人员和程序员通过用户工作流程讨论薄片或线程的图表。如果您与其他地点的团队成员一起工作,请想办法安排电话交谈。如果团队成员难以理解高级测试和需求,您就会知道下次要尝试不同的方法。

You can have all of the diagrams and wiki pages in the world, but if nobody looks at them, they won’t help. Direct communication is always best. Sit down with the programmers and go over the high-level tests and requirements. Go over whiteboard diagrams or paper prototypes together. Figure 17-4 shows a tester and a programmer discussing a diagram of thin slices or threads through a user workflow. If you’re working with a team member in another location, find a way to schedule a phone conversation. If team members have trouble understanding the high-level tests and requirements, you’ll know to try a different approach next time.

图 17-4白板讨论。经 Mike Thomas 许可使用。版权所有 2008。

Figure 17-4 A whiteboard discussion. Used with permission of Mike Thomas. Copyright 2008.

图像

具有良好领域知识的程序员可以立即理解一个故事,甚至可以在编写高级测试之前就开始编码。即便如此,与程序员一起从客户和测试人员的角度审阅故事始终是个好主意。他们对故事的理解可能与您的不同,注意不匹配很重要。记住“三的力量”规则,如果有两种意见无法调和,就抓住客户。测试用例还有助于将故事与应用程序的其余部分联系起来。程序员可以使用测试来帮助他们正确编写故事代码。这是您希望尽可能在迭代开始前完成此操作的主要原因——在程序员开始编码之前。

Programmers with good domain knowledge may understand a story right away and be able to start coding even before high-level tests are written. Even so, it’s always a good idea to review the stories from the customer and tester perspective with the programmers. Their understanding of the story might be different than yours, and it’s important to look at mismatches. Remember the “Power of Three” rule and grab a customer if there are two opinions you can’t reconcile. The test cases also help put the story in context with the rest of the application. Programmers can use the tests to help them to code the story correctly. This is the main reason you want to get this done as close to the start of the iteration as you can—before programmers start to code.

第 2 章,“敏捷测试人员的十大原则”,介绍了“三的力量”规则。

Chapter 2, “Ten Principles for Agile Testers,” introduces the “Power of Three” rule.

不要忘记询问程序员他们认为您可能遗漏了什么。代码的高危区域有哪些?他们认为测试应该集中在哪里?获得更多的技术视角将有助于设计详细的测试用例。如果您已经创建了一个测试矩阵,您可能还想再次查看受影响的区域。

Don’t forget to ask the programmers what they think you might have missed. What are the high-risk areas of the code? Where do they think the testing should be focused? Getting more technical perspective will help with designing detailed test cases. If you’ve created a test matrix, you may want to review the impacted areas again as well.

与程序员一起审查测试的一个有益的副作用是发生的交叉学习。作为测试人员,您会接触到他们的想法,并且他们会学习一些他们在其他情况下不会遇到的测试技术。作为程序员,他们可能会更好地理解他们没有考虑过哪些高级测试。

One beneficial side effect of reviewing the tests with the programmers is the cross-learning that happens. You as a tester are exposed to what they are thinking, and they learn some techniques for testing that they would not have otherwise encountered. As programmers, they may get a better understanding of what high-level tests they hadn’t considered.

测试用例作为文档

Test Cases as Documentation

高级测试用例以及您将在迭代期间编写的可执行测试将构成应用程序文档的核心。在此迭代期间和之后需求会发生变化,因此请确保您的可执行测试用例易于维护。不熟悉敏捷开发的人常常会有没有文档的错误观念。事实上,敏捷项目生成包含可执行测试的可用文档,因此始终是最新的。

High-level test cases, along with the executable tests you’ll write during the iteration, will form the core of your application’s documentation. Requirements will change during and after this iteration, so make sure your executable test cases are easy to maintain. People unfamiliar with agile development often have the misconception that there’s no documentation. In fact, agile projects produce usable documentation that contains executable tests and thus is always up to date.

将可执行测试作为需求文档的一部分的巨大优势在于,很难对它们的结果提出异议。

The great advantage of having executable tests as part of your requirements document is that it’s hard to argue with their results.

丽莎的故事

Lisa’s Story

产品负责人、计划管理员或业务开发经理经常会来问我这样的问题,例如“如果有人提交零美元的贷款付款,系统应该做什么?” 或者“为什么这个计划中的每个人都没有获得 3% 的非选择性贡献?”

Frequently, product owners, plan administrators, or business development managers will come and ask me a question such as, “What’s the system supposed to do if someone submits a loan payment for zero dollars?” or “Why didn’t everyone in this plan get a 3% nonelective contribution?”

向他们展示一个复制场景的 FitNesse 测试比仅仅向他们展示叙述性需求要强大得多。也许系统没有按照应有的方式设计,但测试说明了它实际上是如何工作的,因为我们可以清楚地看到输入和操作的结果。这避免了很多关于“我认为它是这样工作的”级别的争论。

Showing them a FitNesse test that replicates the scenario is much more powerful than just showing them narrative requirements. Maybe the system wasn’t designed the way it should have been, but the test illustrates how it actually works, because we can clearly see the results of the inputs and operations. This has saved a lot of arguments on the level of “I thought it worked this way.”

如果他们认为实现的功能不正确,我们可以更改测试的预期输出并编写一个故事来实现代码,使测试再次通过新的期望。你不能用需求文档来做到这一点。

If they decide the functionality, as implemented, is incorrect, we can change the expected outputs of the test and write a story to implement code to make the test pass again with the new expectations. You can’t do that with a requirements document.

——丽莎

—Lisa

组织测试用例和测试并不总是那么简单。许多团队在 wiki 上记录测试和需求。wiki 的灵活性的缺点是您最终可能会遇到混乱的层次结构。您可能无法找到所需的特定要求或示例。

Organizing the test cases and tests isn’t always straightforward. Many teams document tests and requirements on a wiki. The downside to a wiki’s flexibility is that you can end up with a jumble of hierarchies. You might have trouble finding the particular requirement or example you need.

Lisa 的团队定期重新访问其 wiki 文档和 FitNesse 测试,并重构它们的组织方式。如果您在组织需求和测试用例时遇到困难,请花一些时间研究可能有用的新工具。聘请熟练的技术作家是将您宝贵的测试用例和示例放入易于查找的可用信息库中的好方法。

Lisa’s team periodically revisits its wiki documentation and FitNesse tests, and refactors the way they’re organized. If you’re having trouble organizing your requirements and test cases, budget some time to research new tools that might help. Hiring a skilled technical writer is a good way to get your valuable test cases and examples into a usable repository of easy-to-find information.

第 14 章敏捷测试自动化策略”更多地介绍了测试管理。

Chapter 14, “An Agile Test Automation Strategy,” has more on test management.

概括

Summary

迭代计划会议为整个迭代设定了基调。在本章中,我们了解了敏捷测试人员如何帮助以良好的开端启动迭代。

The iteration planning session sets the tone for the whole iteration. In this chapter, we looked at what agile testers do to help kick off the iteration to a good start.

图像在迭代计划期间,测试人员通过提问和考虑所有观点来帮助团队了解故事。

During iteration planning, testers help the team learn about the stories by asking questions and considering all viewpoints.

图像任务卡需要与开发任务卡一起编写,并根据实际情况进行估算。

Task cards need to be written along with development task cards and estimated realistically.

图像处理测试任务的另一种方法是将它们直接写在开发人员任务卡上。

Another way of tackling testing tasks is to write them directly on the developer task cards.

图像团队应该致力于他们可以完成所有测试任务的工作,因为在完全测试之前没有故事完成。

Teams should commit to the work for which they can complete all of the testing tasks, because no story is done until it’s fully tested.

图像迭代的开始是确保故事可测试并提供足够测试数据的最后机会。

The start of an iteration is the last chance to ensure that the stories are testable and that adequate test data is provided.

图像测试人员与客户协作,详细探索故事并编写高级测试用例,让程序员开始编码。

Testers collaborate with customers to explore stories in detail and write high-level test cases to let programmers kick off coding.

图像测试人员与程序员一起审查高级测试和需求,以确保他们沟通良好。

Testers review high-level tests and requirements with programmers to make sure they are communicating well.

图像测试构成了应用程序文档的核心。

Tests form the core of the application’s documentation.

第 18 章编码和测试

Chapter 18 Coding and Testing

图像

我们的敏捷测试人员帮助规划发布、适当调整故事大小并确保它们是可测试的。她与客户和开发团队的同事一起,将每个故事的期望行为示例转化为高级用户验收测试。她和她的团队已经准备好了实现商业价值所需的资源和基础设施。现在,团队成员已经拿起任务卡开始写代码了。测试人员下一步做什么,尤其是在任何故事准备好测试之前?

Our agile tester has helped plan the release, size stories appropriately, and make sure they’re testable. She, along with colleagues on the customer and development team, has turned examples of desired behavior for each story into high-level user acceptance tests. She and her team have lined up the resources and infrastructure needed to deliver business value. Now, team members have picked up task cards and started writing code. What do testers do next, especially before any stories are ready to test?

推动发展

Driving Development

编码的开始是开始编写详细测试的好时机。在迭代前或迭代前几天编写的高级测试,为程序员开始他们自己的测试驱动开发提供了足够的信息。所以现在我们有一些喘息的空间,但如果我们不迅速行动,编码可能会远远领先于测试并朝着错误的方向发展。

The beginning of coding is a good time to start writing detailed tests. The high-level tests written before the iteration, or in the first couple days of it, provide enough information for the programmers to start their own test-driven development. So now we have a bit of breathing room, but if we don’t move quickly, coding could get way ahead of testing and go off in the wrong direction.

现在是时候开始编写可执行的测试来说明故事的细节,以保持开发的顺利进行并帮助测试与编码保持同步。与高级测试一样,我们根据客户提供的示例进行详细测试。

Now’s the time to start writing executable tests that illustrate the details about a story in order to keep development moving forward smoothly and help testing keep pace with coding. Like the high-level tests, we base detailed tests on examples provided by the customers.

在这一点上,我们主要编写将自动化的测试,但我们也在考虑编码完成后需要进行的重要探索性测试。

At this point, we’re mainly writing tests that will be automated, but we’re also thinking ahead to the important exploratory testing we need to do as coding is completed.

从简单开始

Start Simple

作为测试人员,我们很容易被有趣的代码味道和边缘情况分散注意力。然而,如果我们使用测试来指导编码,我们必须从基础开始。编写最简单的快乐路径测试,以证明核心功能有效。

As testers, we’re easily distracted by interesting code smells and edge cases. However, if we’re using tests to guide coding, we have to start with the basics. Write the simplest happy path test you can in order to show that the core functionality works.

为什么可执行测试?我们的工作时间非常紧,程序员和测试人员都没有时间停下来一遍又一遍地运行手动测试。他们确实有时间单击按钮并运行自动化测试。该测试需要以一种使原因尽可能明显的方式失败。理想情况下,我们会将这些测试提供给程序员,以便他们可以在编写代码时执行它们。这就是为什么选择正确的自动化框架如此重要的原因之一。

Why executable tests? We’re working on an extremely tight schedule, and neither the programmers nor the testers have time to stop and run manual tests over and over. They do have time to click a button and run an automated test. That test needs to fail in a way that makes the cause as obvious as possible. Ideally, we would give these tests to the programmers so that they could execute them as they code. That is one reason why picking the right automation framework is so important.

第 14 章“敏捷自动化策略”给出了选择正确工具的指导。

Chapter 14, “An Agile Automation Strategy,” gives pointers for selecting the right tools.

对于某些故事,自动化测试可能需要很长时间。通过保持第一个测试简单,您可以将重点放在设计自动化解决方案上。当简单的测试有效时,值得花时间进行更复杂的测试用例。

For some stories, automating the tests might take a long time. By keeping the first test simple, you keep the focus on designing the automation solution. When the simple test works, it’s worth putting time into more complex test cases.

我们强调自动化的重要性,但 Janet 曾与成功使用清单或电子表格形式的手动测试的团队合作,为程序员提供他们启动所需的信息。然而,为了从长远来看取得成功,这些测试确实需要自动化。

We stress the importance of automation, but Janet has worked with teams that have successfully used manual tests in the form of checklists or spreadsheets to give the programmers the information they need to start. However, to be successful in the long run, these tests do need to be automated.

增加复杂性

Add Complexity

一旦快乐路径测试起作用,就开始添加更多测试用例。添加边界和边缘条件。测试可能表明程序员误解了需求,或者它们可能表明测试人员误解了需求,或者每个人都没有理解需求的真正含义。重要的是每个人都在谈论它并走上正轨。

As soon as the happy path test works, start adding more test cases. Add boundary and edge conditions. The tests may show that the programmers misunderstood a requirement, or they may show that a tester did, or maybe the requirement’s true meaning eluded everyone. The important thing is that everyone talks about it and gets on track.

当测试人员考虑使用可执行测试验证的新场景时,他们也会考虑手动探索性测试的潜在场景。记下这些以备后用。

As testers think of new scenarios to validate with executable tests, they also think about potential scenarios for manual exploratory testing. Make a note of these for later pursuit.

记住这些测试的目的。他们应该提供示例,告诉程序员要编写什么代码。随着代码的发展,您的测试可以对其提出更多挑战,但要抵制立即将气味跟踪到边缘情况的诱惑。首先让基础知识发挥作用。如果您根据某些风险分析想到更多案例,您以后总是可以添加额外的测试。

Remember the purpose of these tests. They should provide examples that tell the programmers what code to write. As the code evolves, your tests can challenge it more, but resist the temptation to immediately follow smells into edge cases. Get the basics working first. If you think of more cases based on some risk analysis, you can always add extra tests later.

评估风险

Assess Risk

测试人员长期以来一直使用风险分析来帮助确定测试的优先级,并且敏捷开发中已经内置了对风险的考虑。高风险的故事可能会得到更高的规模估计,并且团队在发布和迭代计划期间在优先考虑故事时会考虑风险。

Testers have used risk analysis to help prioritize testing for a long time, and consideration for risk is already built into agile development. High-risk stories may get higher size estimates, and teams consider risk as they prioritize stories during release and iteration planning.

一些快速的风险分析可以帮助您决定首先进行哪些测试以及将精力集中在哪些方面。我们从来没有时间测试所有东西,我们可以使用风险分析来确定多少测试就足够了。

Some quick risk analysis can help you decide what testing to do first and where to focus your efforts. We never have time to test everything, and we can use risk analysis to figure out how much testing is just enough.

如果您的故事非常复杂,您可能希望首先列出与该故事相关的所有潜在风险。这些不限于功能。考虑安全性、性能、可用​​性和其他“能力”。接下来,对于每个项目,使用 1 到 5 的等级(或适合您的任何等级)对每个项目的业务影响进行评级:1 是低影响,5 是严重的负面影响。

If you have a really complex story, you may want to start by listing all of the potential risks related to the story. These aren’t limited to functionality. Consider security, performance, usability, and other “ilities.” Next, for each item, rate the impact on the business if it were to occur, using a scale of 1 to 5 (or whatever scale works for you): 1 being a low impact, 5 being a critical negative impact.

现在,考虑每个项目发生的可能性,使用相同的尺度:1 表示根本不可能发生,5 表示可能会出现的项目。将两个评级相乘以获得每个项目的总风险评级。这使得您可以轻松地挑选出您的团队应该首先关注其测试工作的领域。低风险项目可以留到最后,或者因为它们的影响很低或极不可能发生,所以可能根本不会解决。

Now, consider the likelihood of each item occurring, using the same scale: 1 for not at all likely to happen, and 5 for items that probably will come up. Multiply the two ratings together to get the total risk rating for each item. This makes it easy to pick out the areas where your team should focus its testing efforts first. Low-risk items can be left for last, or, because their impact is low or they’re highly unlikely to occur, may not be addressed at all.

您的域在这里产生了巨大的变化。如果您正在测试在心脏起搏器中运行的软件,您可能需要在测试中涵盖所有风险,无论它们有多低或不太可能。如果您正在测试供一些受过培训的主题专家使用的内部公司 Web 应用程序,您可以跳过不太可能出现或有明显解决方法的场景。

Your domain makes a huge difference here. If you’re testing software that runs in heart pacemakers, you probably need to cover all risks with your testing no matter how low or unlikely they are. If you’re testing an internal company web application to be used by a few trained subject matter experts, you may be able to skip over scenarios that are unlikely or have an obvious workaround.

考虑图 18-1中的故事。

Consider the story in Figure 18-1.

图 18-1关于运输速度的故事

Figure 18-1 Story on shipping speeds

图像

图 18-2显示了此运输成本案例的可能风险评估。

Figure 18-2 shows a possible risk assessment for this shipping cost story.

表 18-2风险评估样本

Table 18-2 Sample risk assessment

图像

项目 8 是风险最高的项目,因此我们希望确保测试更改的送货地址并验证更新后的成本。我们可能想要自动化这个场景的端到端测试。我们不太担心第 5 项;也许我们已经测试了我们的邮政编码验证并且感觉良好,所以我们不需要再测试它。您甚至可能有一个您选择不测试的风险非常低的项目。

Item 8 is the highest-risk item, so we’d want to be sure to test changing shipping addresses and verify the updated costs. We might want to automate an end-to-end test with this scenario. We’re not too worried about item 5; maybe we have already tested our postal code validation and feel good about it, so we don’t need to test it more. You may even have a very low-risk item that you chose not to test.

历史通常是一位好老师。记下过去的问题并确保它们不会再次发生。

History is usually a good teacher. Take note of past issues and make sure they don’t happen again.

编码和测试共同进步

Coding and Testing Progress Together

在迭代的这一点上,编码和测试继续携手并进。测试人员、程序员、数据库专家和其他团队成员按照示例和测试提供的指南协作开发故事。不同的团队成员可能会贡献他们的特殊专业知识,但他们都觉得有责任确保每个故事都完成。随着工作的进展,他们都了解了这个故事并互相学习。

At this point in the iteration, coding and testing continue hand in hand. Testers, programmers, database experts, and other team members collaborate to develop the stories, following the guidelines provided by examples and tests. Different team members may contribute their particular expertise, but all of them feel responsible for making sure each story is finished. All of them learn about the story and learn from each other as work progresses.

让我们看看团队如何处理图 18-1中的运输成本故事。Patty 程序员拿起一张任务卡来对估计的运费计算进行编码。她已经从早期的讨论中很好地理解了这个故事,但她可能会查看 wiki 页面或故事卡片的背面,测试人员在这些地方写下了一些描述故事目的的叙述,一些故事应该如何运作的例子,以及一些高级测试,以确保她清楚地知道从哪里开始。Tammy Tester 看到编码工作已经开始,并开始编写用于成本计算的 GUI 测试用例。

Let’s look at how a team might work on the shipping cost story in Figure 18-1. Patty Programmer picks up a task card to code the estimated shipping cost calculations. She already understands the story pretty well from earlier discussions, but she may look at the wiki pages or back of the story card where the testers wrote down some narrative describing the purpose of the story, some examples of how it should work, and some high-level tests to make sure she has a good idea of where to start. Tammy Tester sees that coding work has begun and starts to write behind-the-GUI test cases for the cost calculations.

该团队在计划期间同意首先根据送货地址和物品重量计算 5 天的送货成本。项目只能在北美大陆内运送,但验证将在表示层完成,因此成本计算测试可以假设仅考虑有效目的地作为输入。他们正在使用运输合作伙伴提供的成本计算 API,Tammy 询问 Patty 在哪里可以找到算法,以便她可以自己计算成本以编写测试。Tammy 在他们的 behind-the-GUI 测试工具中编写了她能想到的最简单的测试用例。我们在图 18-3中将其显示为一个简单的表格。

The team had agreed during planning to start by calculating the 5-day shipping cost based on the shipping address and item weight. Items can only be shipped within continental North America, but that validation will be done in the presentation layer, so the cost calculation tests can assume only valid destinations are considered for input. They’re using a cost calculation API provided by the shipping partner, and Tammy asks Patty where to find the algorithms so she can figure the cost herself in order to write the tests. Tammy writes the simplest test case she can think of in their behind-the-GUI test tool. We show it as a simple table in Figure 18-3.

表 18-3简单快乐路径测试

Table 18-3 Simple happy path test

图像

Patty 尚未完成使该测试通过的代码,因此 Tammy 开始着手该故事的另一项测试任务,设置测试环境以与运输合作伙伴的测试系统一起工作。

Patty hasn’t finished the code that would make this test pass yet, so Tammy starts working on another testing task for the story, setting up the test environment to work with the shipping partner’s test system.

识别变化

Identify Variations

因为这个故事和测试非常简单,所以 Patty 和 Tammy 没有像讨论更复杂的故事那样讨论测试设计和调整它。他们也不需要向产品负责人询问更多问题。Patty 打电话给 Tammy,让她知道这个简单的测试现在可以正常工作了。Tammy 编写了更多测试用例,在美国尝试不同的权重和目的地。这些都很好。她尝试了加拿大的邮政编码,但测试出现异常。她向帕蒂展示了这一点,帕蒂意识到 API 默认为美国邮政编码,并且需要加拿大和墨西哥代码的国家/地区代码。她还没有为其他国家编写任何单元测试。他们修改测试输入,Patty 与 Paul Programmer 结对更改调用 API 的代码。现在测试看起来像图 18-4

Because this story and the test are so straightforward, Patty and Tammy don’t discuss the test design and tweak it as they might on more complex stories. They also haven’t needed to ask the product owner more questions yet. Patty calls Tammy over to show her that the simple test is now working. Tammy writes up more test cases, trying different weights and destinations within the United States. Those all work fine. She tries a Canadian postal code, and the test gets an exception. She shows this to Patty, who realizes that the API defaults to U.S. postal codes, and requires a country code for codes in Canada and Mexico. She hadn’t written any unit tests yet for other countries. They revise the test inputs, and Patty pairs with Paul Programmer to change the code that calls the API. Now the test looks something like Figure 18-4.

表 18-4修改后的快乐路径测试

Table 18-4 Revised happy path test

图像

这个简单的例子说明了编码和测试之间的反复来回。不同的团队采取不同的方法。Patty 和 Tammy 可能会在编码和测试方面结对。Tammy 可能会与 Paul 合作编写 fixture 来自动化测试。Tammy 可能在远程办公室,使用在线协作工具与 Patty 一起工作。Patty 可能会自己编写可执行的故事测试,然后编写代码使它们工作,实践真实的故事测试驱动开发。关键是测试和编码是所有团队成员参与的一个开发过程的一部分。

This simple example illustrates the iterative back-and-forth between coding and testing. Different teams take different approaches. Patty and Tammy might pair on both the coding and testing. Tammy might pair with Paul to write the fixture to automate the test. Tammy might be in a remote office, using an online collaboration tool to work with Patty. Patty might write the executable story tests herself and then write the code to make them work, practicing true story test-driven development. The point is that testing and coding are part of one development process in which all team members participate.

Tammy 可以继续识别新的测试用例,包括边缘用例和边界条件,直到她觉得所有风险区域都已被最少数量和种类的测试用例覆盖。她可能会用网站上最重的商品发送到最昂贵的目的地进行测试。她可能会尝试大量购买相同的物品。一些极端情况可能不太可能发生,以至于她不去理会它们,或者她决定运行测试,但在测试通过后没有将其包含在回归套件中。某些测试最好在 UI 可用后手动完成。

Tammy can continue to identify new test cases, including edge cases and boundary conditions, until she feels all risk areas have been covered by the minimum amount and variety of test cases. She might test with the heaviest item available on the website sent to the most expensive destination. She might test having a large quantity of the same item. Some edge cases may be so unlikely that she doesn’t bother with them, or she decides to run a test but after it passes doesn’t include it in the regression suite. Some tests might be better done manually after a UI is available.

三的力量

Power of Three

Patty 编写了将夏威夷作为运输目的地的单元测试,但 Tammy 认为只有大陆目的地是可以接受的。他们都不确定军事邮政信箱目的地是否可以接受。他们去见 Polly 产品负责人,询问她的想法。他们正在使用三的力量。当出现分歧或问题时,拥有三个不同的观点是确保您获得良好解决方案并且以后不必重新讨论问题的有效方法。如果参与讨论的人中有一个不熟悉这个话题,其他人就必须整理思路,把它解释清楚,这总是有帮助的。让不同角色的人参与进来有助于确保对需求的更改不会悄悄发生,并在以后让团队成员感到惊讶。

Patty has written unit tests with Hawaii as the shipping destination, but Tammy believes that only continental destinations are acceptable. Neither of them is sure whether military post office box destinations are acceptable. They go see Polly Product-Owner to ask what she thinks. They’re using the Power of Three. When disagreements or questions arise, having three different viewpoints is an effective way to make sure you get a good solution and you won’t have to rehash the issue later. If one of the participants in the discussion isn’t familiar with the topic, the others will have to organize their thoughts to explain it clearly, which is always helpful. Involving people in different roles helps make sure that changes to requirements don’t fly under the radar and surprise team members later.

当意想不到的问题出现时,三的力量规则是一个很好的起点。根据问题的严重性或复杂性,您可能需要召集更多人,甚至是整个团队。如果运输合作伙伴的 API 被证明太慢以至于网站上的响应时间无法接受怎么办?开发团队和客户团队都需要快速探索替代解决方案。

When unexpected problems arise, as they always do, the Power of Three rule is a great place to start. You may need to pull in more people, or even the whole team, depending on the severity or complexity of the issue. What if the shipping partner’s API proves to be so slow that the response time on the website will be unacceptable? Both the development team and the customer team need to quickly explore alternative solutions.

专注于一个故事

Focus on One Story

Paul 正在寻找要处理的编程任务。虽然预估运费故事的 UI 任务仍在任务板上的“待办事项”栏中,但他对从购物车中删除商品的故事更感兴趣,因此他拿起其中一张卡片。没有人有时间开始为那个故事编写可执行测试,所以他自己一头扎了进去。

Paul looks for a programming task to work on. Although the UI tasks for the estimated shipping cost story are still in the “to do” column on the task board, he’s more interested in the story to delete items out of the shopping cart, so he picks up one of those cards. Nobody has time to start writing the executable tests for that story, so he plunges ahead on his own.

现在团队有两个故事。他们真的不知道完成这两个故事需要多少时间。一个更好的方法是让 Paul 开始处理第一个故事的 UI 任务,这样故事就可以更快地完成。当一个故事完成时(意味着所有代码都已编写和测试),您确切地知道还有多少工作要做:零。如果灾难来袭并且没有其他故事在本次迭代中完成,则至少有一个完整的故事要发布。

Now the team has two stories going. They don’t really know how much time it will take to finish either story. A much better approach would be for Paul to start working on a UI task for the first story so that story can be finished sooner. When a story’s done (meaning all of the code is written and tested), you know exactly how much work is left to do on it: zero. If disaster struck and no other stories got finished this iteration, there is at least one completed story to release.

完成整个故事不是测试概念,而是测试人员应该提倡和遵循的概念。如果程序员已经开始编写故事,请确保有人也开始为该故事进行测试任务。这是一种平衡行为。如果没有人为删除项目故事编写过高级测试怎么办?也许这是最高的测试优先级?通常,完成一个故事应该是团队可以继续下一个故事之前的目标。

Completing the whole story isn’t a testing concept, but it’s one that testers should promote and follow. If a programmer has started coding on a story, make sure someone has also started working on testing tasks for that story. This is a balancing act. What if nobody has written even high-level tests for the delete items story? Maybe that’s the highest testing priority? Usually, finishing a story should be the goal before the team can move on to the next story.

除非团队非常小,否则在任何给定时间总是有不止一个故事在进行中。这可能会更困难,但请尝试一次专注于完成一个故事。Patty 即将结束关于运费的故事,而 Paul 已经开始讨论删除项目的故事。帕蒂遇到了障碍,她不知道该如何解决。Paul 帮助她完成代码,以便 Tammy 可以完成她的探索性测试,他们可以将故事标记为“完成”。现在他们可以更好地了解完成本次迭代还有多少工作要做(或者至少,还有多少工作是他们不需要处理的)。

Unless the team is very small, there is always more than one story in progress at any given time. It might be more difficult, but try to focus on finishing one story at a time. Patty is about to wrap up the shipping cost story, and Paul has moved on to the delete items story. Patty runs into a snag, and she isn’t sure how to solve it. Paul helps her to finish the code so that Tammy can finish her exploratory testing and they can mark the story “done.” Now they have a better idea of how much they have left to finish this iteration (or at least, how much they don’t still have to work on).

有时,如果程序员和测试人员结对一起完成每个故事,则可以同时完成几个不同的故事。如果故事很小且独立,则此方法有效。您不希望看到的是程序员在没有同时完成测试任务的情况下开始编码。

Sometimes, several different stories can be done at the same time if a programmer and tester pair up to complete each story together. This works if the stories are small and independent. What you don’t want to see is programmers starting coding without testing tasks being completed at the same time.

批评产品的测试

Tests that Critique the Product

一旦可测试的代码块可用,并且指导其编码的自动化测试通过,就需要时间更深入地探索功能。尝试不同的场景并了解有关代码行为的更多信息。您应该有用于批评产品的测试的任务卡,包括面向业务和技术的。在完成所有这些类型的测试之前,故事还没有“完成”。

As soon as testable chunks of code are available, and the automated tests that guided their coding pass, take time to explore the functionality more deeply. Try different scenarios and learn more about the code’s behavior. You should have task cards for tests that critique the product, both business- and technology-facing. The story’s not “done” until all of these types of tests are complete.

当故事的所有任务(测试除外)都已完成时,这一点变得更加重要。现在您应该能够从故事线索的一端到另一端进行测试,并测试其间的所有变化。不要推迟这个测试。您可能会发现需求在故事中,但在推动开发的测试中遗漏了,因此在代码中遗漏了。现在是编写那些缺失的测试和代码的时候了。在团队仍然专注于故事的同时,填补所有空白并增加更多价值。以后这样做会花费更多。

This becomes more important when all tasks except testing are complete for a story. Now you should be able to test from one end of the story’s thread to the other end, with all of the variations in between. Don’t put this testing off. You may find requirements that were in the story but were missed with the tests that drove development and are thus missing in the code. Now’s the time to write those missing tests and code. Fill in all of the gaps and add more value while the team is still focused on the story. Doing this later will cost much more.

请注意,您在测试最终故事中学到的一些东西可能被认为是“拥有的很好”,可能使功能更易于使用或更快,这些项目不是原始故事的一部分。咨询您的客户。如果有时间在迭代中添加它,并且业务可以使用额外的价值,那就继续吧。现在添加这些添加物要便宜得多。但是不要因为花太多时间添加投资回报率不高的“金光闪闪”而危及其他故事。

Be aware that some of what you learn in testing the final story may be considered “nice to have,” perhaps making the functionality easier to use or faster, items that weren’t part of the original story. Consult with your customer. If there’s time to add it in the iteration, and the business can use the extra value, go ahead. These additions are much cheaper to add now. But don’t jeopardize other stories by spending too much time adding “bling” that doesn’t have a big ROI.

第 10 章面向业务的产品评判测试”和第 11 章“面向技术的评测产品的测试”将帮助您确保涵盖评判产品的所有必要测试。

Chapter 10, “Business-Facing Tests that Critique the Product,” and Chapter 11, “Tecnology-Facing Tests that Critique the Product,” will help you make sure you cover all of the necessary tests that critique the product.

如果您的探索性测试让团队和客户意识到故事未涵盖重要功能,请为未来的迭代编写新故事。严格控制“范围蔓延”,否则您的团队将没有时间交付您最初计划的价值。

If your exploratory testing leads the team and the customers to realize that significant functionality wasn’t covered by the stories, write new stories for future iterations. Keep a tight rein on “scope creep” or your team won’t have time to deliver the value you planned originally.

评价产品的面向技术的测试通常最好在编码期间完成。现在是了解设计是否无法扩展或是否存在安全漏洞的时候了。

Technology-facing tests to critique the product are often best done during coding. This is the time to know if the design doesn’t scale, or if there are security holes.

与程序员合作

Collaborate with Programmers

我们的小插图描述了一个团队编写和使用详细的测试来驱动编码,展示了测试人员和程序员的密切协作。随着编码和测试的进行,这种情况会继续。一起工作可以增强团队交付正确产品的能力,并提供许多传授技能的机会。程序员学习新的测试方法,他们会在编写代码时更好地测试自己的代码。测试人员更多地了解编码过程以及正确的测试如何使编码过程变得更容易。

Our vignette describing a team writing and using detailed tests to drive coding shows how closely testers and programmers collaborate. This continues as coding and testing proceed. Working together enhances the team’s ability to deliver the right product and provides many opportunities to transfer skills. Programmers learn new ways of testing, and they’ll be better at testing their own code as they write it. Testers learn more about the process of coding and how the right tests might make it easier.

结对测试

Pair Testing

Paul Programmer 已经完成了估计运输选项故事的用户界面,但他还没有签入。他请 Tammy 过来和他坐在一起,并演示最终用户如何在结帐过程中输入送货地址。估计的运费立即显示。Tammy 更改了送货地址并看到新的费用出现。她输入的邮政编码与地址的其余部分不匹配,并看到相应的错误消息出现。UI 对他们俩来说都不错,因此 Paul 签入了代码,而 Tammy 继续对其进行探索性手动测试。

Paul Programmer has completed the user interface for the estimated shipping options story, but he hasn’t checked it in yet. He asks Tammy to come sit with him and demonstrates how the end user would enter the shipping address during the checkout process. The estimated shipping cost displays right away. Tammy changes the shipping address and sees the new cost appear. She enters a postal code that doesn’t match the rest of the address and sees the appropriate error message appear. The UI looks good to both of them, so Paul checks in the code, and Tammy continues with her exploratory manual testing of it.

珍妮特喜欢让程序员在这些结对测试期间“开车”,同时她会观察发生的情况。她发现这比在程序员注视下控制键盘和鼠标要有效得多。

Janet likes to have the programmer “drive” during these pair testing sessions while she watches what happens. She finds that it is far more effective than taking control of the keyboard and mouse while the programmer watches.

“给我看看”

“Show Me”

Tammy 特别关心更改送货地址和重新计算估计成本,因为他们将其确定为风险区域。她发现,如果她显示估算费用,然后转到账单地址页面,然后回来更改送货地址,则估算费用不会正确更改。她让保罗来观察这种行为。他意识到会话缓存存在问题并返回修复它。

Tammy is especially concerned with changing the shipping address and having the estimated cost recalculate, because they identified that as a risky area. She finds that if she displays the estimated cost, goes ahead to the billing address page, and then comes back to change the shipping address, the estimated costs don’t change properly. She gets Paul to come observe this behavior. He realizes there is a problem with session caching and goes back to fix it.

向某人展示问题并一起解决它比在缺陷跟踪系统中提交错误并等待某人有时间查看它要有效得多。如果团队不在同一地点,就更难做到。如果团队成员在截然不同的时区工作,这就更难了。坚持最直接的沟通方式。丽莎的一位队友所在的时区比现在早 12.5 小时。他工作到深夜,在需要时,他会打电话给丽莎,他们一起研究测试结果和示例。

Showing someone a problem and working through it together is much more effective than filing a bug in a defect tracking system and waiting for someone to have time to look at it. It’s harder to do if the team isn’t co-located. If team members are working in vastly different time zones, it’s even harder. Stick to the most direct communication available to you. One of Lisa’s teammates is in a time zone 12½ hours ahead. He works late into his nighttime, and when needed, he calls Lisa and they work through test results and examples together.

向另一个人展示 GUI 的简单行为可能会帮助 Paul 意识到他实施了一些错误的行为。同样,如果 Tammy 在使她的 GUI 测试脚本工作时遇到问题,解释这个问题可能足以让她意识到是什么原因造成的。如果没有人可以查看您刚刚编写的代码或帮助您调试问题,有时向您自己大声解释它会有所帮助。“橡皮鸭”和“大声思考”是解决你自己问题的非常有效的方法。珍妮特喜欢把她自己的小橡皮鸭放在她的桌子上,以提醒自己在提问之前要三思。

The simple act of showing the GUI to another person may help Paul realize he’s implemented some erroneous behavior. Similarly, if Tammy is having trouble getting her GUI test script to work, explaining the problem might be enough for her to realize what’s causing it. If there is nobody available to look at what you’ve just coded or help you debug a problem, it sometimes helps to explain it out loud to yourself. “Rubber Ducking” and “Thinking Out Loud” are surprisingly effective ways to solve your own problems. Janet likes to have her own little rubber duck sitting on her desk to remind herself to think before she asks.

参考书目包含进一步阅读该主题的参考资料。

The bibliography contains references for further reading on this subject.

与客户交谈

Talk to Customers

对于开发团队成员来说,埋头编造故事而忘记让客户了解情况是非常容易的。除了在遇到问题时咨询业务专家外,我们还需要向他们展示我们目前已交付的成果。

It’s shockingly easy for development team members to get their heads down cranking out stories and forget to keep customers in the loop. In addition to consulting business experts when we have questions, we need to show them what we’ve delivered so far.

希望在编码开始之前,您能够与客户或可以代表客户的人一起审查测试用例。如果没有,永远不会太晚。对于客户需要更多地参与可执行测试细节的情况,一定要找到适合他们和技术团队成员的测试工具。

Hopefully, you were able to review test cases with customers, or with someone who could represent the customer, before coding began. If not, it’s never too late. For situations where customers need to be more involved with the details of the executable tests, be sure to find test tools that work for them as well as for technical team members.

正如我们在上两章中所描述的,您可能已经与客户讨论过实体模型或纸质原型。如果模拟报告或界面的任务保留在迭代计划中,请记住保持过程简单。例如,不要编写 HTML 原型,而在白板上绘图也可以。我们希望流程尽可能简单;简单是一个核心价值。

As we described in the last two chapters, you may have already gone over mock-ups or paper prototypes with your customers. If tasks to mock up a report or interface remain in the iteration plan, remember to keep the process simple. For example, don’t code an HTML prototype when drawing on a whiteboard will do just as well. We want to keep the process as simple as possible; simplicity is a core value.

显示客户

Show Customers

一旦编码的用户界面或报告准备就绪,即使它仍然很初级,缺少所有功能或显示硬编码数据,也要将其展示给适当的客户。没有人能提前准确解释他们想要什么。他们需要通过查看、感受和使用应用程序来了解它是否正确。你可能不是能够在迭代中期实施大的改变,但如果你早点开始,可能会有时间进行小的调整,你的客户就会知道会发生什么。

As soon as a coded user interface or report is ready, even if it’s still rudimentary, lacking all features or displaying hard-coded data, show it to the appropriate customers. Nobody can explain exactly what they want ahead of time. They need to see, feel, and use the application to know if it’s right. You may not be able to implement big changes mid-iteration, but if you start early, there may be time for minor tweaks, and your customers will know what to expect.

迭代评审会议是一个很好的机会来展示团队交付了什么并获得下一次迭代的反馈,但不要等到那时才从客户那里获得输入。让他们参与整个迭代。

The iteration review meeting is a great opportunity to show what the team delivered and get feedback for the next iteration, but don’t wait until then to get input from customers. Keep them involved throughout the iteration.

了解业务

Understand the Business

虽然我们陷入了快速迭代的节奏,但我们也需要停下来花时间更好地了解业务。花一些时间与业务人员讨论他们的工作以及哪些方面可以通过新软件功能得到增强。您越了解客户的业务,就越能提供优质的产品。

Although we get caught up in the fast pace of iterations, we also need to stop and take time to understand the business better. Spend some time talking to business people about their jobs and what aspects might be enhanced with new software features. The better you understand your customer’s business, the better you can be at providing a good product.

丽莎的故事

Lisa’s Story

我的团队为每个开发团队成员安排时间,让他们在日常工作中与退休计划管理团队成员坐在一起。我们不仅更好地了解这些工作,而且我们经常发现应用程序中的小变化,这些变化将使管理员的工作更轻松。

My team budgets time for each development team member to sit with the retirement plan administration team members as they do their daily work. Not only do we understand those jobs better, but we often identify small changes in the application that will make the administrator’s work easier.

简单的添加(例如提供的一些额外数据、额外的搜索过滤器或更改显示的顺序)可以使繁琐而详细的过程大不相同。我们还用流程图和 wiki 页面记录我们学到的东西,以便其他团队成员可以受益。

Simple additions such as a bit of extra data provided, an additional search filter, or changing the order of a display can make a big difference to a tedious and detailed process. We also document what we learn with flow charts and wiki pages so that other team members can benefit.

——丽莎

—Lisa

有些团队实际上长期与业务人员坐在一起,以便他们每天都参与实际业务。

Some teams actually sit with the business people permanently so that they are involved with the actual business on a daily basis.

完成测试任务

Completing Testing Tasks

敏捷测试人员是主动的。我们不会坐等工作来找我们。习惯于瀑布流程的测试人员可能会觉得在故事 100% 完成之前无事可做。在敏捷迭代期间,这很少是真的。与程序员一起工作,以便他们尽早生成一些可测试的代码。前面介绍的运输成本算法就是一个很好的例子。它可以完全独立地进行测试,无需访问数据库或用户界面。或者,可以在访问真实数据的服务完成之前用硬编码数据去除用户界面,并且可以自行测试表示层的行为。

Agile testers are proactive. We don’t sit and wait for work to come to us. Testers who are accustomed to a waterfall process may feel there’s nothing to do until a story is 100% complete. That’s rarely true during an agile iteration. Work with programmers so that they produce some testable piece of code early on. The shipping cost algorithm presented earlier is a good example. It can be tested completely in isolation, without needing to access the database or the user interface. Alternatively, the user interface could be stubbed out with hard-coded data before the services accessing the real data are complete, and the behavior of the presentation layer can be tested by itself.

危险:测试紧缩

Peril: The Testing Crunch

即使是经验丰富的敏捷团队也经常在迭代结束时经历测试紧缩。也许一两个故事比预期花费的时间长得多,或者制作问题占用了开发时间。当明天是您的迭代结束并且您的任务板(真实的或虚拟的)仍然充满测试卡时会发生什么?

Even experienced agile teams often experience a testing crunch at the end of an iteration. Maybe a story or two turned out to take much longer than expected, or a production problem took time away from development. What happens when tomorrow is the end of your iteration and your task board (real or virtual) is still full of testing cards?

如果您看到这种情况,请将其识别为难闻的气味。与团队一起确定可能是什么问题。程序员与测试人员的合作不够紧密吗?有太多的干扰吗?

If you see this, recognize it as a bad smell. Work with the team to determine what the problem may be. Are the programmers not working closely enough with the testers? Were there too many interruptions?

解决这一危险的方法是让整个团队参与进来。请记住,团队中的任何人都可以报名参加测试任务。在每日站会中,您可以评估团队是否按计划完成所有故事。如果多个故事有未完成的危险,请选择要删除的故事,或缩小一个或多个故事的范围。专注于一次完成一个故事。随着迭代接近尾声,程序员可能不得不停止开发新功能,转而开始接受测试任务。缺少一个版本中的某些功能比缺少整个版本要好,因为无法对所有或大多数故事完成测试。

The way to address this peril is to involve the whole team. Remember that anyone on the team can sign up for testing tasks. In your daily stand-up, you can evaluate whether the team is on track to finish all of the stories. If multiple stories are in danger of not being completed, choose a story to drop, or reduce the scope on one or more stories. Focus on completing one story at a time. As the end of the iteration approaches, programmers may have to stop working on new features and start picking up testing tasks instead. Missing some functionality from a release is better than missing the entire release because testing couldn’t be completed on all or most stories.

除了单元和集成测试之外,Lisa 团队的程序员还定期自动执行后台 GUI 测试。他们还经常编写功能性的 GUI 测试用例。有时他们会编写初始的快乐路径可执行测试,以便他们可以协调测试和代码设计;然后测试人员添加更多测试用例。有时,他们会编写所有的功能测试用例,因为测试人员没有足够的带宽来涵盖所有的测试密集型故事。

The programmers on Lisa’s team regularly automate behind-the-GUI tests in addition to unit and integration tests. They also often write the functional behind-the-GUI test cases. Sometimes they write the initial happy path executable test so they can coordinate test and code design; then a tester adds more test cases. Occasionally, they write all of the functional test cases, because the testers don’t have the bandwidth to cover all of the test-intensive stories.

团队中的每个人还必须愿意承担手动测试任务。如果您的团队刚刚起步并且还不能解决自动化需求,那么整个团队应该计划时间来执行手动回归测试脚本以及手动测试新功能。正如 Lisa 的团队可以证明的那样,这项任务为学习如何设计应用程序以促进测试自动化提供了巨大的动力。其他团队告诉我们这对他们也有效。

Everyone on the team also must be willing to take on manual testing tasks. If your team is just starting and hasn’t been able to address automation needs yet, the whole team should plan time to execute manual regression test scripts as well as manually testing new features. As Lisa’s team can attest, this task provides great motivation for learning how to design the application to facilitate test automation. Other teams tell us this worked for them as well.

处理错误

Dealing with Bugs

我们知道许多团队都在为如何跟踪错误或是否要跟踪错误的问题而苦苦挣扎。正如 Tom 和 Mary Poppendieck 在他们的书《实施精益软件开发:从概念到现金》[2006] 中所写,缺陷队列是返工队列,因此是浪费。有些团队一发现错误就简单地修复它们。他们编写一个单元测试来重现错误,修复代码以使测试通过,签入测试和错误修复,然后继续。如果稍后有人破坏了那段代码,测试将捕获回归。

We’ve known many teams that struggle with the question of how to track bugs, or whether to track them at all. As Tom and Mary Poppendieck write in their book Implementing Lean Software Development: From Concept to Cash [2006], defect queues are queues of rework and thus collection points for waste. Some teams simply fix bugs as soon as they’re discovered. They write a unit test to reproduce the bug, fix the code so the test passes, check in the test and the bug fix, and go on. If someone breaks that piece of code later, the test will catch the regression.

其他团队发现在缺陷跟踪系统 (DTS) 中记录问题和修复很有价值,尤其是在代码发布后才发现的问题。他们甚至可能会在生产中的错误中寻找模式,并进行根本原因分析以了解如何防止类似问题再次发生。尽管如此,缺陷系统并没有提供一个关于如何生成更高质量代码的面对面交流的良好论坛。

Other teams find value in documenting problems and fixes in a defect tracking system (DTS), especially problems that weren’t caught until after code was released. They may even look for patterns in the bugs that got to production and do root cause analysis to learn how to prevent similar issues from recurring. Still, defect systems don’t provide a good forum for face-to-face communication about how to produce higher-quality code.

第 5 章,“转换典型流程”,讨论您的团队可能想要或不想使用缺陷跟踪系统的原因。

Chapter 5, “Transitioning Typical Processes,” talks about why your team may or may not want to use a Defect Tracking System.

Lisa 和她的测试伙伴们更愿意在发现问题后立即与程序员交谈。如果程序员可以立即修复它,则无需在任何地方记录错误。如果没有程序员可以立即处理该问题,并且有可能忘记该错误,他们会为它写一张卡片或将其输入到他们的 DTS 中。

Lisa and her fellow testers prefer to talk to a programmer as soon as a problem is found. If the programmer can fix it immediately, there’s no need to log the bug anywhere. If no programmer is available immediately to work on the problem, and there’s a possibility the bug might be forgotten, they write a card for it or enter it into their DTS.

我们将此部分添加到本章,因为这是您遇到问题的时候。您一直在先编写测试,但在与程序员合作时发现了问题。你记录错误吗?如果是这样,如何?您一直在进行探索性测试,并从标记为完成的故事中发现了一个错误。您是否为此记录了错误?让我们更多地讨论缺陷并考虑对您和您的团队开放的选项。

We’ve added this section to this chapter because this is when you run into the problem. You have been writing tests first, but are finding problems as you work with the programmer. Do you log a bug? If so, how? You’ve been doing your exploratory testing and found a bug from a story that was marked done. Do you log a bug for that? Let’s discuss more about defects and consider options that are open to you and your team.

它是缺陷还是特性?

Is It a Defect or Is It a Feature?

首先,让我们谈谈缺陷与特性。软件开发中的一个古老问题是,“什么是错误”?我们听到的一些答案是:偏离要求或行为不符合预期。当然,也有一些非常明显的缺陷,例如不正确的输出或不正确的错误消息。但真正重要的是用户对产品质量的看法。如果客户说这是一个缺陷,那么它就是一个缺陷。

First, let’s talk about defects versus features. The age-old question in software development is, “What is a bug”? Some answers we’ve heard are: It’s a deviation from the requirements or it’s behavior that is not what was expected. Of course, there are some really obvious defects such as incorrect output or incorrect error messages. But what really matters is the user’s perception of the quality of the product. If the customer says it is a defect, then it is a defect.

在敏捷中,我们有机会与客户合作,让他们满意地解决问题。客户不必事先考虑所有可能的功能和细节。当他们看到某些东西时改变主意对他们来说是可以的。

In agile, we have the opportunity to work with customers to get things fixed to their satisfaction. Customers don’t have to try to think of every possible feature and detail up front. It is okay for them to change their minds when they see something.

最后,如果需要修复它是错误还是功能真的很重要吗?客户选择优先事项和价值主张。如果软件质量对客户来说比获得所有新功能更重要,那么我们应该尝试修复我们发现的所有缺陷。

In the end, does it really matter if it is a bug or a feature if it needs to be fixed? The customer chooses priorities and the value proposition. If software quality is a higher priority for the customer than getting all of the new features, then we should try to fix all defects as we find them.

团队中的客户利用他们的知识在日常开发中为团队提供最好的建议。然而,当产品进入 UAT 并暴露给更大的客户群时,总会有以错误或新增强形式出现的请求。

Customers on the team use their knowledge to give the best advice they can to the team on day-to-day development. However, when a product goes to UAT and is exposed to a larger customer base, there will always be requests in the form of bugs or new enhancements.

技术债务

Technical Debt

考虑缺陷的一种方式是技术债务。缺陷在系统中停留且未被发现的时间越长,影响就越大。确实,让错误在代码库中恶化会对代码质量、系统直观性、系统灵活性、团队士气和速度产生负面影响。修复错误代码中的一个缺陷可能会揭示更多,因此维护任务需要更长的时间。

One way of thinking about defects is as technical debt. The longer a defect stays in the system and goes undetected, the greater the impact. It also is true that leaving bugs festering in a code base has a negative effect on code quality, system intuitiveness, system flexibility, team morale, and velocity. Fixing one defect in buggy code may reveal more, so maintenance tasks take longer.

第 6 章,“测试的目的”,解释了测试如何帮助管理技术债务。

Chapter 6, “The Purpose of Testing,” explains how tests help manage technical debt.

零错误容忍

Zero Bug Tolerance

Janet 鼓励与她合作的团队努力实现对错误数量的“零容忍”。新的敏捷团队通常很难相信它可以完成。在 Janet 工作的一个组织中,她向五个项目团队中的每一个提出挑战,看看他们在每次迭代结束时如何接近零未解决的错误,并在发布时达到零错误。

Janet encourages teams that she works with to strive for “zero tolerance” toward bug counts. New agile teams usually have a hard time believing it can be done. In one organization Janet was working with, she challenged each of the five project teams to see how close they could come to zero bugs outstanding at the end of each iteration, and zero at release time.

作为任何开发的一部分,您总是需要做出权衡。您的团队可能决定发布带有一些突出错误的版本,因为人们认为推出新功能比修复低级错误更重要。

As part of any development, you will always need to make trade-offs. Your team may decide to release with some outstanding bugs because it is deemed more important to get new functionality out the door than to fix low-level bugs.

一切都与选择有关

It’s All about Choices

团队已经解决了如何以多种不同方式处理缺陷的问题。一些团队将所有错误都放在任务卡上。其他团队选择写一张卡片,对其进行估算,并将其安排为一个故事。还有一些人建议为每个错误添加一个测试——这样您就不必记录缺陷,只需记录测试。

Teams have solved the problem of how to handle defects in many different ways. Some teams put all of their bugs on task cards. Other teams have chosen to write a card, estimate it, and schedule it as a story. Still others suggest adding a test for every bug—that way you don’t have to record the defect, just the test.

有一种正确的方法吗?当然不是!但是,您怎么知道什么适合您的团队?我们有一些建议可以帮助您选择和决定什么是适合您的。想想你的团队和你的产品,以及什么可能适合你的情况。首先,我们将讨论我们应该记录哪些缺陷,然后我们将讨论何时应该修复它们,最后我们将看看选择什么媒体。正确的组合将取决于您的团队在敏捷之旅中走了多远以及您的产品有多成熟。

Is there one right way? Of course not! But, how do you know what is right for your team? We have some suggestions to help you choose and decide what is right for you. Think about your team and your product and what might work in your situation. First, we’ll talk about what defects we should log, then we’ll talk a bit about when you should fix them, and finally we’ll look at what media to choose. The right combination will depend on how far along your team is in its agile journey and how mature your product is.

决定记录哪些错误

Decide Which Bugs to Log

并非所有错误都需要记录,但团队经常纠结哪些应该记录,哪些不需要记录。我们建议您尽可能避免创建缺陷报告。首先与真人交谈,只有在确实是需要更改产品或程序员无法立即解决的真正问题时才生成缺陷报告。

Not all bugs need to be logged, but teams often struggle with which ones should be recorded and which ones don’t need to be. We recommend that you avoid creating a defect report if possible. Have a conversation with a real person first, and only produce a defect report if it is truly a real problem that demands a change to the product or the programmers just can’t get to it right away.

单元测试失败

不要记录单元测试失败。如果您所在的团队正在实践 TDD(测试驱动开发)并且对其单元测试有很好的覆盖率,那么您就会知道构建期间失败的测试不应该被记录下来。在持续集成构建期间失败的测试是程序员立即解决问题的信号。记录这些错误是多余的,而且是浪费时间。

Don’t log unit test failures. If you are part of a team that is practicing TDD (test-driven development) and has good coverage with its unit tests, you know that failed tests during the build should not be logged. A failed test during the continuous integration build is a signal for the programmers to address the problem right away. Logging these bugs would be redundant and a waste of time.

更高级别的回归测试失败

许多团队都有在单元级别之上运行回归测试的构建,例如 GUI 后面的测试和通过 GUI 进行的测试。当其中一个构建失败时,您是否应该在 DTS 中记录错误?

Many teams have builds that run regression tests above the unit level, such as tests behind the GUI and tests through the GUI. When one of these builds fails, should you log the bug in a DTS?

丽莎的故事

Lisa’s Story

我们有两个构建,一个只运行单元测试的“正在进行的构建”,以及一个在 GUI 后面并通过 GUI 运行功能测试的“完整构建”。当“完整构建”中断时,如果开发人员像有时发生的那样立即调查并解决问题,通常不会记录错误。问题很快就解决了。在其他时候,失败并不简单。其中一名测试人员进行调查,缩小问题范围,并提交一个错误,说明失败测试的名称或提供手动步骤来重现问题。

We have two builds, an “ongoing build” that runs only unit tests, and a “full build” that runs the functional tests behind and through the GUI. When the “full build” breaks, if a developer investigates and tackles the problem right away as sometimes happens, usually no bug is logged. The problem is fixed quickly. At other times, the failure is not straightforward. One of the testers investigates, narrows down the problem, and files a bug that either states the name of the failing test or provides manual steps to recreate the problem.

在任何一种情况下,都会编写重现错误的测试,并修复代码以使测试通过。测试成为其中一个构建的一部分。

In either case, tests are written that reproduce the bug, and the code is fixed to make the tests pass. The tests become part of one of the builds.

——丽莎

—Lisa

失败的测试本身就是一种记录错误。但有时,就像 Lisa 的情况一样,需要添加更多信息才能进行有效且干净的修复,因此记录缺陷是有必要的。

Failing tests in themselves are a type of recorded bug. But sometimes, as in Lisa’s case, more information needs to be added to allow for an effective and clean fix, so logging the defect is warranted.

当前迭代中的故事错误

不要记录可以立即修复的错误,特别是如果您要以其他方式将它们记录在电子 DTS 中。如果您的团队与程序员,并在故事完成后立即进行配对测试,我们强烈建议您不要记录这些错误,只要程序员立即解决它们即可。当您注意到问题时,请与程序员讨论并确定它们是否是真正的问题。如果需要,请与客户交谈,但做一些笔记,以便记住您所看到的内容,以便在需要时调整您的测试。

Don’t log bugs that can be fixed immediately, especially if you would otherwise record them in an electronic DTS. If your team is working closely with the programmers and is practicing pair testing as soon as a story is completed, we strongly recommend that you don’t log those bugs as long as the programmer addresses them right away. As you notice issues, talk them over with the programmer and decide whether they are real issues or not. Talk to the customer if you need to, but make a couple of notes so you remember what you saw so you can adjust your tests if needed.

如果您使用索引卡来记录错误,您可能需要将索引卡放在任务板上(或电子板上的卡片)以作为提醒。

If you are using index cards to log bugs, you may want to put an index card up on the task board (or a card on your electronic board) just as a reminder.

迭代后错误(或无法立即修复的错误)

记录无法立即修复的错误。我们尽早强调测试,以便在程序员仍在处理故事时捕获尽可能多的错误。我们知道在早期发现时修复它们会更便宜;但是,有时我们只是没有立即抓住它们。程序员已经转向另一个故事,现在不能放下一切来修复它。这些是日志记录的良好候选者。有时,“错误”实际上是一个遗漏的需求,需要作为一个故事来处理——为未来的迭代进行估计和优先级排序。

Do log bugs that can’t be fixed right away. We stress testing early in order to catch as many bugs as possible while the programmers are still working on the story. We know it is cheaper to fix them when caught early; however, sometimes we just don’t catch them right away. The programmer has moved on to another story and can’t drop everything to fix it now. Those are the ones that are good candidates for logging. Sometimes a “bug” is really a missed requirement and needs to be handled as a story—estimated and prioritized for a future iteration.

来自遗留系统

记录遗留系统中发生的错误。如果您的产品已经存在很长时间,它可能有许多潜伏在后台等待被发现的错误。当您找到它们时,您有几个选择。如果你的产品负责人认为修复它们是值得的,那么记录这些错误,它们可以作为产品待办事项列表的一部分进行优先排序。但是,如果它们已经存在了很长时间并且没有引起任何问题,您的产品负责人可能会认为不值得修复它们。在这种情况下,不要费心记录它们。他们无论如何都不会得到解决,所以不要浪费你的时间。

Do log bugs that occur in the legacy system. If your product has been around a long time, it likely has a number of bugs that have been lurking in the background just waiting to be discovered. When you find them, you have a couple of choices. If your product owner thinks it is worthwhile to fix them, then log the bugs and they can be prioritized as part of the product backlog. However, if they have been around a long time and cause no issues, your product owner may decide it is not worth fixing them. In this case, don’t bother logging them. They will never get addressed anyhow, so don’t waste your time.

在生产中发现

记录所有生产错误。当您的应用程序投入生产时,应记录客户发现的所有错误。根据它们的严重程度,这些错误可能会在下一个版本发布时立即修复,或者将对它们进行估计、确定优先级并放入您的产品待办事项列表中。

Do log all production bugs. When your application is in production, all bugs found by the customer should be logged. Depending on their severity, these bugs may be fixed immediately, at the time of the next release, or they’ll be estimated, prioritized, and put in your product backlog.

选择何时修复错误

Choose When to Fix Your Bugs

共有三个选项。您发现的所有错误都需要进行分类,以确定您是现在修复它们、稍后修复它们,还是根本不修复它们。这种分类可能就像与程序员讨论以确定它们是否真的一样简单他正在研究的故事中的错误。分类可能是与产品所有者的讨论,以确定下一次迭代是否应该有另一个故事。分类也可能是与客户确定优先修复哪些错误的正式过程。

There are three options. All bugs you find need to be triaged to determine if you fix them now, fix them later, or don’t fix them at all. This triage may be as simple as a discussion with the programmer to determine if they are really bugs in the story he is working on. The triage may be a discussion with the product owner to determine if there should be another story for the next iteration. The triage may also be a formal process with the customers to prioritize which bugs to fix.

现在就修

您可以立即修复的错误越多,您的应用程序产生的技术债务就越少,您拥有的“缺陷”库存就越少。越早发现缺陷,修复它们的成本也就越低。在iSixSigma Magazine的一篇文章中,Mukesh Soni [2008] 引用了 IBM 的一份报告,指出修复产品发布后发现的错误的成本是设计期间发现的错误的四到五倍,比识别的错误高出 100 倍在维护阶段(见图18-5)。

The more bugs you can fix immediately, the less technical debt your application generates and the less “defect” inventory you have. Defects are also cheaper to fix the sooner they are discovered. In an article in iSixSigma Magazine, Mukesh Soni [2008] quotes a report from IBM that the cost to fix an error found after product release was four to five times as much as one uncovered during design, and up to 100 times more than one identified in the maintenance phase (see Figure 18-5).

图 18-5修复软件缺陷的相对成本

Figure 18-5 Relative costs to fix software defects

图像

图 18-5显示了基于分阶段方法的统计数据,但该统计数据仍然适用于敏捷开发。修复在开发过程中发现的错误比在开发之后更便宜。

Figure 18-5 shows a statistic based on phased methodology, but the statistic still holds true for agile development. It is cheaper to fix bugs that are found during development than after.

如果在开发新功能时发现缺陷,或者是另一个错误修复的副作用,则应该自动修复。但是,像往常一样,这要谨慎应用。例如,如果发现程序员说很难修复并且可能会破坏产品稳定性的错误,则应将其带给客户以确定优先级。

If a defect is found while developing a new feature, or is a side effect from another bug fix, it should be automatically fixed. But, as usual, this is to be applied with prudence. For example, if a bug is found that the programmers say will be difficult to fix and may destabilize the product, it should be taken to the customers to prioritize.

如果您在开发过程中修复错误,则可以减少该过程后期出现的错误。您的团队速度可以包括修复错误的时间。随着时间的推移,您的团队成员会很好地了解他们花了多长时间来修复测试人员为故事发现的错误。希望很少。如果你的团队是一个新的敏捷团队,可能会有相当多的错误逃避开发,但随着团队对工具和流程越来越熟悉,发现的错误数量将会减少。首先,尝试对一个故事进行估算,以包括两个小时或半天的时间来修复相关的错误。

If you fix the bugs during development, you lessen the presence of bugs later in the process. Your team velocity can include time to fix bugs. Over time, your team members will get a good idea of how long they spend on fixing bugs found by the testers for a story. Hopefully, there are few. If your team is a new agile team, there may be quite a few bugs that escape development, but as the team gets more comfortable with the tools and the processes, the number of bugs found will lessen. To start, try making the estimate for a story to include two hours or half a day for fixing associated bugs.

稍后修复

不同的团队有不同的处理缺陷的方法。一些团队认为,所有发现的缺陷在被列入修复清单之前应该由客户确定优先级。他们认为完全由客户来决定它们是否真的是缺陷,如果是,是否应该修复它们。

Different teams have different ways of handling defects. Some teams believe that all defects found should be prioritized by the customers before they get put on the list to fix. They believe it is completely up to the customer to determine whether they really are defects, and if so, whether they should be fixed.

永不修复

您的团队已经识别出一个缺陷,但知道它不会得到修复。也许那部分代码以后需要完全重写,因为功能会发生变化,或者它可能只是一个低优先级的问题或太模糊以至于您的客户可能永远找不到它。无法修复的原因有很多。如果您的分类确定是这种情况,我们建议您关闭该错误。不要假装有一天你会修好它。

Your team has recognized a defect, but know it won’t get fixed. Perhaps that section of code needs a complete rewrite later because the functionality will change, or perhaps it is just such a low-priority issue or so obscure that your customers may never find it. There are a multitude of reasons why it won’t get fixed. If your triage determines this is the case, we suggest you just close the bug. Don’t keep it open pretending that you will fix it someday.

选择您应该用来记录错误的媒体

Choose the Media You Should Use to Log a Bug

当我们谈论媒体时,我们指的是记录错误的各种方式。它可以是缺陷跟踪系统或索引卡,或者您可能选择根本没有物理记录。

When we talk about media, we mean the variety of ways you can log a bug. It could be a defect tracking system or index cards, or maybe you choose to have no physical record at all.

索引卡

索引卡(无论是在线计划和跟踪系统中的真实卡还是虚拟卡)不会为大量文书细节留出太多空间,但是当它们固定在故事板上时,它们确实可以很好地显示未决问题,尤其是如果它们是另一种颜色。一些团队使用屏幕打印并将它们钉在卡片背面或将详细信息写在文本文件中,甚至在手持录音机上以音频形式记录步骤。

Index cards (whether real or virtual cards in an online planning and tracking system) don’t leave a lot of room for a lot of clerical details, but they do give great visibility to outstanding issues when they are pinned on the story board, especially if they are in another color. Some teams use screen prints and staple them to the back of the card or write the details in a text file, or even record steps in audio form on a hand-held voice recorder.

有很多选项,但我们建议您选择一个包含足够信息的选项,以指导某人重现问题或在程序员准备修复问题时集中讨论。卡片是有形的。DTS 中的五百个错误只是一个数字。一叠 500 张卡片令人印象深刻。

There are lots of options, but we would suggest that you pick one that contains enough information to guide someone to reproduce a problem or to focus a discussion when the programmer is ready to fix it. The card is tangible. Five hundred bugs in a DTS are just a number. A stack of 500 cards is impressive.

在以下情况下使用卡片:

Use cards in the following circumstances:

图像您是一个纪律严明的敏捷团队,并且在迭代中修复所有错误。

You are a disciplined agile team and are fixing all bugs within an iteration.

图像您想让团队看到错误。

You want to make bugs visible to the team.

没有什么能阻止您同时拥有索引卡和 DTS。

There is nothing stopping you from having both index cards and a DTS.

缺陷追踪系统

在以下情况下使用 DTS:

Use a DTS in the following circumstances:

图像你的团队是分布式的。

Your team is distributed.

图像您需要出于审计目的跟踪错误或在发行说明中捕获它们。

You need to track bugs for audit purposes or to capture them in release notes.

图像您有逃脱迭代的错误,您需要记住稍后修复它们。

You have bugs that escape an iteration and you need to remember to fix them later.

图像您的遗留系统存在大量缺陷。

You have a legacy system with a large number of defects.

不管怎样,您可能希望使用某种 DTS 来记录某些错误。这并不意味着您需要将它们全部记录下来。明智地选择记录哪些内容。

One way or the other, you will likely want to have some kind of DTS to log some of the bugs. This does not mean you need to log them all. Be smart about which ones you do log.

一个都没有

你为什么不记录错误?与我们合作过的大多数团队都为自己设定了规则,即没有单元测试就无法修复错误。如果您还有一个功能性自动化套件,那么您可以用它们来捕获更大的错误。论点是,如果有一个测试可以捕获错误,则无需记录错误。从修复错误中学到的任何东西都被记录在测试和代码中。但是,您需要认识到并非所有测试都易于自动化。

Why wouldn’t you log a bug? Most teams that we have worked with have set rules for themselves that no bug is fixed without a unit test. If you also have a functional automation suite, then you can catch the larger bugs with those. The argument is that if there is a test that will catch the bug, you have no need to log the bug. Anything learned from fixing the bug was captured in the test and the code. However, you need to recognize that not all tests are easy to automate.

在以下情况下使用测试捕获错误:

Use tests to capture bugs in the following circumstance:

图像您的团队纪律严明,并为发现的每个错误编写测试。

Your team is disciplined and writes tests for every bug found.

处理错误的替代方案和建议

Alternatives and Suggestions for Dealing with Bugs

随着团队的成熟,他们会找到适合他们的程序。他们消除了多余的任务。他们更加熟练地使用故事卡、故事董事会和项目积压。他们有效地使用测试,并了解要记录哪些错误以及哪些指标对他们的团队有意义。在本节中,我们将分享其他团队为他们找到的一些想法。

As teams mature, they find procedures that work for them. They eliminate redundant tasks. They become more practiced at using story cards, story boards, and project backlogs. They use tests effectively, and learn which bugs to log and what metrics make sense to their team. In this section, we’ll share some ideas that other teams have found work for them.

设定规则

设定规则,例如“粉红色卡片(错误)的数量在任何时候都不应超过十张。” 每次进行团队回顾时都要重新审视这些。如果您的缺陷率正在下降,请不要担心。如果趋势相反,请花时间分析错误的根本原因并创建新规则来缓解这些错误。

Set rules like, “The number of pink cards (bugs) should never get higher than ten at any one time.” Revisit these each time you have a team retrospective. If your defect rate is going down, no worries. If the trend is the opposite, spend time analyzing the root cause of bugs and create new rules to mitigate those.

修复所有错误

不要忘记修复在迭代过程中发现的低优先级错误,因为它们会影响未来的开发。根据我们的经验,“低优先级”和“快速修复”之间似乎存在很强的相关性,尽管我们没有确凿的事实来支持这一点。我们建议在小的、孤立的错误变成大的、纠结的错误之前阻止它们。

Don’t forget to fix low-priority bugs found during the iteration as well, because they have an effect on future development. In our experience, there seems to be a strong correlation between “low priority” and “quick to fix,” although we don’t have hard facts to support that. We suggest stopping small, isolated bugs before they become large, tangled bugs.

合并错误

如果您在一个区域中发现了很多错误,请考虑将它们组合成一个增强功能或故事。

If you find a lot of bugs in one area, think about combining them into an enhancement or story.

珍妮特的故事

Janet’s Story

当我刚开始在西捷工作时,我发现移动应用程序存在很多小问题。该应用程序运行正常,但我对流程感到困惑。我只是发现这些问题,因为我是新手,没有以前的看法。

When I first started working at WestJet, I found a lot of small issues with the mobile application. The application worked correctly, but I was confused about the flow. I only found these issues because I was new and had no previous perceptions.

团队决定将我提出的问题归为一组,并将整个问题视为一个新故事。在研究了所有已知细节的完整问题之后,最终结果是一个可靠的特征。如果这些错误是逐步修复的,那么效果就不会那么漂亮。

The team decided to group the issues I had raised and look at the whole issue as a new story. After studying the full problem with all of the known details, the final outcome was a solid feature. If the bugs had been fixed piecemeal, the effect would not have been so pretty.

——珍妮特

—Janet

把它当作一个故事

如果一个“错误”确实是错过的功能,选择为错误写一张卡片并将其安排为一个故事。就像任何其他故事一样,对这些故事进行估计和优先级排序。请注意,错误故事可能不会像产品待办事项列表中的新用户故事那样受到关注。创建故事、确定优先级和安排时间也需要时间。

If a “bug” is really missed functionality, choose to write a card for the bug and schedule it as a story. These stories are estimated and prioritized just like any other story. Be aware that bug stories may not receive as much attention as the new user stories in the product backlog. It also takes time to create the story, prioritize, and schedule it.

如果错误只是简单地记录在 DTS 中,那么重要信息可能会从项目中丢失。当我们编写验收测试来驱动开发时,我们倾向于关注所需的行为。从缺陷中了解不良行为并将其转化为故事是生成正确功能的重要补充。

If bugs are simply logged in a DTS, important information might be effectively lost from the project. When we write acceptance tests to drive development, we tend to focus on desired behavior. Learning about undesired behavior from a defect, and turning that into stories is a vital addition to producing the right functionality.

蓝色、绿色和红色贴纸

每个团队都需要确定适合自己的流程,以及如何使该流程易于查看。下面的故事是关于一个对珍妮特有效的过程。

Each team needs to determine the process that works for it, and how to make that process easily visible. The following story is about one process that worked for Janet.

珍妮特的故事

Janet’s Story

几年前,我在一个遗留系统上工作,在引入敏捷之前,该系统已经记录了许多错误。其中一位开发人员坚决表示他不会使用缺陷跟踪系统。他坚信他们是浪费时间。但是,测试人员需要记录缺陷,因为缺陷太多了。

A few years ago, I worked on a legacy system with lots of bugs already logged against the system before agile was introduced. One of the developers was adamant that he would not use a defect-tracking system. He firmly believed they were a waste of time. However, the testers needed the defects logged because there were so many.

该团队制定了对每个人都有效的折衷方案。在与程序员进行配对测试期间发现的错误没有记录下来,因为它们会立即得到修复。所有其他人都记录在 DTS 中。需要在当前迭代中修复的错误记录在带有摘要和错误编号的粉红色卡片上,然后放在故事板上。所有其他的都成为产品积压的一部分。

The team worked out a compromise that worked for everyone. Bugs that were found during pair testing with the programmers were not recorded, because they were fixed right away. All others were logged in the DTS. Bugs that needed to be fixed in the current iteration were recorded on pink cards with the summary and bug number and then put on the story board. All others became part of the product backlog.

程序员可以查看系统中的详细信息,但如果需要,也可以向测试人员询问更多信息。因为问题在故事板上,所以它们成为日常站立和讨论的一部分。修复错误后,程序员会在卡片背面写下修复和任何额外信息。他们在卡上贴了一张蓝色贴纸,这样测试人员就知道它已经准备好进行测试了。绿色标签表示它已被验证为已修复,红色标签表示它未修复且需要更多工作。当然,测试人员和程序员之间有很多对话。James,其中一名程序员,和我一起从一个无法修复的 bug 中获得了很多乐趣。到最后,这张卡片看起来就像上面有一条毛毛虫——蓝色、红色、蓝色、红色、蓝色,最后是绿色。当这个错误被解决时,我们都非常兴奋。

The programmers could look at details in the system but also asked testers for more information, if required. Because the issues were on the story board, they became part of the daily stand-ups and discussions. When a bug was fixed, the programmers wrote the fix and any extra information on the back of the card. They put a blue sticker on the card so the testers knew it was ready for testing. A green sticker meant it had been verified as fixed, and a red sticker meant it wasn’t fixed and needed more work. Of course, there were lots of conversations between the testers and the programmers. James, one of the programmers, and I had a lot of fun with one bug that just wouldn’t stay fixed. By the end, the card looked like it had a caterpillar on it—blue, red, blue, red, blue, and finally green. We were all quite excited when that bug was squashed.

测试人员关闭了错误并完成了大部分管理工作,因为 DTS 是他们的要求。一段时间后,程序员开始将他们修复的内容输入缺陷跟踪系统,因为这比写在卡片上更容易。由于可见性,团队仍然继续使用这些卡片。很容易一眼看出迭代或积压中有多少未解决的错误。

The testers closed bugs and did most of the administration, because the DTS was their requirement. After a while, the programmers started entering what they fixed into the defect-tracking system because it was easier than writing on the card. The team still continued to use the cards because of the visibility. It was easy to see at a glance how many outstanding bugs there were in the iteration or on the backlog.

——珍妮特

—Janet

这种方法对这个团队很有效,因为团队中有很多纪律,而且大多数新错误如果是新功能或更改功能的一部分,就会在迭代中得到修复。唯一进入积压工作的错误是被认为是低风险的遗留错误。

This approach worked for this team because there was a lot of discipline in the team, and most new bugs were fixed in the iteration if they were part of the new or changed functionality. The only bugs that went into the backlog were legacy bugs that were deemed low risk.

从简单开始

Start Simple

我们建议使用尽可能简单的系统,并根据需要应用复杂性。根据我们的经验,测试优先生成的代码在签入时基本上没有错误。如果您在新代码中发现很多错误,您的团队需要找出原因并采取行动。尽量缩短编码、集成和测试的周期,让程序员得到有关代码质量的即时反馈。也许遗留代码的某些错误部分需要重新设计,以免让您的团队陷入技术债务。也许您需要与业务专家更紧密地合作以了解所需的功能。

We suggest using as simple a system as possible and applying complexity as required. Code produced test-first is, in our experience, fairly free of bugs by the time it’s checked in. If you’re finding a lot of bugs in new code, your team needs to figure out why, and take action. Try to shorten the cycle of coding, integrating and testing so that programmers get immediate feedback about code quality. Perhaps some buggy section of legacy code needs to be redesigned before it mires your team in technical debt. Maybe you need to work more closely with the business experts to understand the desired functionality.

另一个想法可能是创建一个持续的“开始、停止、继续”列表,以便您可以在迭代回顾期间记住一些问题。

Another idea might be to create an ongoing “start, stop, continue” list so that you can remember some of the issues during the iteration retrospective.

有关回顾的更多信息,请参见第 19 章结束迭代”。

More on retrospectives in Chapter 19, “Wrap Up the Iteration.”

促进沟通

Facilitate Communication

每日站会帮助团队保持他们需要的密切沟通。团队中的每个人都了解任务和故事的当前状态,并且可以互相帮助克服障碍。通常,听到程序员描述他们正在处理的任务会提供一个线索,表明他们可能误解了客户的需求。这表明需要在站立后进行小组讨论。如果测试人员需要帮助解决出现的测试问题,她可能会要求团队在站会后留下来讨论。遗漏的任务通常会在站立时被识别出来,并且可以当场写新卡片。

The daily stand-up helps teams maintain the close communication they need. Everyone on the team learns the current status of tasks and stories, and can help each other with obstacles. Often, hearing programmers describe tasks they’re working on provides a clue that they may have misunderstood the customer’s requirements. That signals the need for a group discussion after the stand-up. If a tester needs help with a testing issue that’s come up, she might ask the team to stay after the stand-up to talk about it. Missed tasks are often identified during stand-ups, and new cards can be written on the spot.

站会是查看进展情况的好时机。使用大的、可见的图表,例如故事板、燃尽图和其他视觉提示,以帮助保持专注并了解您的状态。如果迭代的末尾临近,并且故事的编码似乎“停滞不前”,请举起红旗并询问团队可以做些什么。也许一些配对或额外的帮助会让事情顺利进行。Lisa 经常注意到还有很多测试要做而时间不多了。她请求帮助收拾残局。整个团队专注于完成每个故事需要做什么,并讨论最佳方法。

The stand-up is a good time to look at progress. Use big, visible charts such as story boards, burndown charts, and other visual cues to help keep focus and know your status. If the end of the iteration is drawing near, and coding on a story seems “stuck,” raise a red flag and ask the team what can be done about it. Perhaps some pairing or extra help will get things going. Lisa has often noted when there’s a lot of testing left to do and time is running out. She asks for help to pick up the slack. The whole team focuses on what needs to be done to complete each story and talks about the best approach.

当团队使用电子媒体来跟踪故事时,往往会忘记故事板。珍妮特发现同时拥有这两者似乎是重复工作,但团队取得进展的可见性远远超过编写任务卡并在完成时移动它们的额外开销。拥有故事板可以让您的团队在站立会议期间或在您与团队外的人谈论您的进度时集中注意力。

When teams use an electronic medium for keeping track of stories, there is a tendency to forget the story board. Janet finds that having both may seem like a duplication of effort, but the visibility of progress to the team far outweighs the extra overhead of writing up the task cards and moving them as they are completed. Having the story board gives your team focus during the stand-ups or when you are talking to someone outside the team about your progress.

测试人员促进沟通

Testers Facilitate Communication

测试人员可以通过帮助确保每个人都进行足够的沟通来帮助保持迭代的顺利进行。当程序员开始编写故事时与他们交谈,并确保他们理解它。Lisa 发现她可以在团队 wiki 上编写她想要的所有测试和示例,但如果没有人愿意阅读它们,它们就无济于事。如有疑问,她会与拿起任务卡的程序员一起检查需求和测试。

Testers can help keep the iteration progressing smoothly by helping make sure everyone is communicating enough. Talk to programmers when they start working on a story, and make sure they understand it. Lisa finds that she can write all of the tests and examples she wants on the team wiki, but if nobody bothers to read them, they don’t help. When in doubt, she goes over requirements and tests with the programmer who picks up the task cards.

程序员在开发故事时总会有问题,即使他们非常了解业务和故事。如果有客户最好回答问题,因为那是最直接的沟通。测试人员不应该妨碍它;然而,我们观察到业务专家有时难以解释需求,或者程序员只是理解错误,无法与客户达成共识。三的力量适用于此。测试人员可以帮助客户和程序员找到共同语言。

Programmers will always have questions as they develop a story, even if they understand the business and the story well. It’s best if a customer is available to answer questions, because that is the most direct communication. Testers shouldn’t get in the way of that; however, we’ve observed that business experts sometimes have trouble explaining a requirement, or a programmer simply gets the wrong idea and can’t get on the same page with the customer. The Power of Three applies here. Testers can help customers and programmers find a common language.

促进沟通通常涉及在白板上绘图、模拟界面、列出可能受影响的其他区域或通过实际示例进行操作。每当沟通似乎进入死胡同,或混乱猖獗时,要求一个新的例子并专注于此。

Facilitating communication usually involves drawing on a whiteboard, mocking up interfaces, listing other areas that might be affected, or working through real examples. Whenever communication appears to reach a dead end, or confusion is rampant, ask for a new example and focus on that.

丽莎的故事

Lisa’s Story

当退休计划参与者想要从他们的账户中提取资金时,许多复杂的归属规则和政府法规就会发挥作用。如果参与者过去曾取款,情况会变得更糟。在编写一个故事来计算参与者的既得余额时,我的团队成员对正确的算法都有不同的想法,即使产品所有者已经完成迭代开始时的几个例子。我的测试伙伴迈克要求产品负责人完成一个新示例,几位程序员和测试人员加入了会议。在白板上写数字和流程图花了几个相当曲折的时间,但最终他们得出了正确的公式,而且每个人都在同一页上。

When retirement plan participants want to withdraw money from their accounts, many complex vesting rules and government regulations come into play. It gets worse if the participant has withdrawn money in the past. Working on a story to calculate a participant’s vested balance, my team members all had different ideas on the correct algorithm, even though the product owner had worked through several examples at the beginning of the iteration. My fellow tester, Mike, asked the product owner to work through a new example, and several programmers and testers joined the session. It took a couple of rather tortuous hours of writing numbers and flowcharts on a whiteboard, but eventually they arrived at the correct formula, and everyone was on the same page.

——丽莎

—Lisa

根据需要完成尽可能多的示例,直到团队充分了解系统的不同方面。如果不起作用,请尝试其他格式。例如,如果画在白板上的图片不足以理解故事,请尝试使用电子表格或业务专家熟悉的其他格式。

Work through as many examples as you need until the team understands enough different aspects of the system. Try a different format if it’s not working. For example, if pictures drawn on the whiteboard aren’t sufficient to understand the story, try spreadsheets or some other format that’s familiar to the business experts.

分布式团队

Distributed Teams

正如我们在其他章节中提到的,团队成员位于不同地点和不同时区意味着您必须更加努力地进行沟通。电话、电子邮件和即时消息构成了通信的基础,但更好的协作工具一直在开发。

As we’ve noted in other chapters, having team members in different locations and different time zones means you have to work harder at communication. Phones, email, and instant messaging form the basics of communication, but better collaboration tools are developed all the time.

第 9 章,“支持团队的面向业务的测试工具包”,讨论了一些可以帮助分布式团队的工具

Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” talks about some tools that can help distributed teams

丽莎的故事

Lisa’s Story

我们团队中的一名程序员,同时也是一名经理,搬到了印度。Nanda 在那里工作到深夜,所以他早上可以为丹佛队效力。他有一部带有本地丹佛电话号码的手机,因此很容易通过电话以及即时消息和电子邮件与他交谈。我们安排在一大早就讨论故事的会议,例如评估会议、头脑风暴会议和迭代计划,这样他就可以参加。尽管团队无法像我们在同一地点工作时那样高效,但我们仍然能够受益于 Nanda 的领域专业知识和对软件的深入了解。

One of the programmers on our team, who is also a manager, moved to India. Nanda works late into the evening there, so he’s available for the Denver team in the mornings. He has a cell phone with a local Denver phone number, so it’s easy to talk to him by phone as well as by instant message and email. We schedule meetings where we discuss stories, such as estimating meetings, brainstorming sessions, and iteration planning, early in the morning so he can participate. Although the team can’t be as productive as we were when we were co-located, we’re still able to benefit from Nanda’s domain expertise and deep knowledge of the software.

如果 Nanda 在印度雇佣更多的团队成员,我们可能不得不解决更复杂的问题,比如协调集成和构建。我们可能会考虑更复杂的技术解决方案来解决通信问题。

If Nanda hires more team members in India, we may have to address more complex issues, such as coordinating integration and builds. We may consider more sophisticated technical solutions to communication problems.

——丽莎

—Lisa

您将需要进行试验,看看什么对您的分布式团队有效。使用回顾来评估协作和沟通是否需要改进,并集思广益以改进方法。作为一名测试人员,您可能在帮助流程改进项目方面拥有丰富的经验。只需考虑将改善沟通作为持续改进需求之一。

You will need to experiment to see what works for your distributed team. Use retrospectives to evaluate whether collaboration and communication need improving, and brainstorm ways to improve. You, as a tester, may have a lot of experience in helping with process improvement projects. Just think about improving communication as one of those continual improvement needs.

为了我们的项目取得成功,我们都需要能够相互良好沟通。当团队位于不同的地理位置时,他们可能需要加倍努力才能保持联系。

We all need to be able to communicate well with each other for our projects to succeed. When teams are in diverse geographic locations, they might have to work twice as hard to stay in constant touch.

回归测试

Regression Tests

除非你在一个刚刚开始其自动化工作的团队中,否则你有覆盖以前迭代故事的自动化回归测试。希望这些作为持续构建过程的一部分运行,或者至少作为日常构建过程的一部分运行。如果他们不是,请让您的团队将实施这个关键基础设施作为优先事项,并与他们一起集思广益如何做到这一点。计划下一次迭代的时间以开始构建过程。

Unless you’re on a team that’s just starting its automation efforts, you have automated regression tests covering stories from previous iterations. Hopefully, these are running as part of a continual build process, or at least part of a daily build process. If they aren’t, ask your team to make implementing this critical infrastructure a priority, and brainstorm with them how this might be done. Plan time in the next iteration to start a build process.

保持构建“绿色”

Keep the Build “Green”

程序员应该在签入新代码之前运行所有自动化单元测试。但是,单元测试可能会在持续构建中失败,要么是因为有人在签入之前忘记运行它们,要么是因为运行时环境或 IDE 不同。我们进行单元测试是有原因的,所以每当一个失败时,团队的最高优先级(除了 showstopper 生产问题)应该是修复它并让构建再次运行。

Programmers should run all automated unit tests before checking in new code. However, unit tests may fail in the continual build, either because someone forgot to run them before check-in, or because of a difference in runtime environment or IDE. We have unit tests for a reason, so whenever one fails, the team’s highest priority (apart from a showstopper production issue) should be to fix it and get the build working again.

团队采用不同的方法来确保他们的构建保持“绿色”。Lisa 的团队有一个构建过程,每次构建后都会通过电子邮件发送结果。如果构建失败,签入失败的人通常会立即修复它。如果不清楚构建失败的原因,团队成员将聚在一起进行调查。他们的 ScrumMaster 有一个毛绒玩具,她把它放在“破坏构建”的人的桌子上,作为视觉提醒,它必须立即修复。

Teams take different approaches to make sure their build stays “green.” Lisa’s team has a build process that emails results after every build. If the build fails, the person who checked in the failure usually fixes it right away. If it’s not clear why the build failed, team members will get together to investigate. Their ScrumMaster has a stuffed toy that she puts on the desk of the person who “broke the build,” as a visual reminder that it has to be fixed right away.

一些团队使用红绿灯、环境球体、GUI 构建监控工具或其他电子视觉方式来显示构建状态。当灯变红时,就该停止新开发并修复构建了。另一种技术是在每个人的 IDE 中弹出一个屏幕,显示构建失败,并且弹出窗口不会消失,直到您单击“Ok,我会修复构建”。享受它的乐趣,但保持构建运行是一项严肃的工作。

Some teams use a traffic light, ambient orb, GUI build monitoring tool, or other electronic visual way to show the build status. When the lights turn red, it’s time to stop new development and fix the build. Another technique is to have a screen pop up in everyone’s IDE showing that the build has failed, and the popup won’t go away until you click “Ok, I’ll fix the build.” Have some fun with it, but keeping the build running is serious business.

在极端情况下,您可能不得不暂时注释掉一个失败的测试,直到它可以被诊断出来,但这是一种危险的做法,尤其是对于新手团队而言。如有必要,团队中的每个人都应该停止他们正在做的事情,直到构建再次运行。

In extreme cases, you may have to temporarily comment out a failing test until it can be diagnosed, but this is a dangerous practice, especially for a novice team. Everyone on the team should stop what they’re doing if necessary until the build works again.

保持快速构建

Keep the Build Quick

构建需要提供即时反馈,因此请保持简短。如果构建花费的时间超过代码签入的平均频率,构建就会开始堆积,测试人员无法获得他们需要测试的代码。XP 的构建时间指南是十分钟 [ Fowler, 2006 ]。Lisa 的团队试图将构建时间控制在八分钟以内,因为他们经常检查。

The build needs to provide immediate feedback, so keep it short. If the build takes longer than the average frequency of code check-ins, builds start to stack up, and testers can’t get the code they need to test. The XP guideline for build time is ten minutes [Fowler, 2006]. Lisa’s team tries to keep the build less than eight minutes, because they check in so often.

花费时间过长的测试,例如更新数据库的测试、单元级别以上的功能测试或 GUI 测试脚本,应该在单独的构建过程中运行。如果团队的硬件有限,他们可能不得不在晚上运行带有全套测试的“完整”构建,以及在工作时间持续运行只有单元测试的“持续”构建。拥有一个单独的、持续的“完整”构建以及所有回归测试套件是值得投资的。Lisa 的团队每 90 分钟从他们的“完整”构建中获得反馈,这已经事实证明,它在解决回归问题方面具有无可估量的价值。这个辅助测试套件不会阻止程序员检查他们的代码。

Tests that take too long, such as tests that update the database, functional tests above the unit level, or GUI test scripts, should run in a separate build process. If the team is limited in hardware, they might have to run the “full” build with the full suite of tests at night and the “ongoing” build that has only unit tests continually during working hours. Having a separate, continual “full” build with all of the regression test suites is worth the investment. Lisa’s team gets feedback every 90 minutes from their “full” build, and this has proven invaluable in heading off regression issues. This secondary suite of tests does not stop a programmer from checking in their code.

构建回归套件

Building a Regression Suite

在迭代期间,您正在自动化新测试。一旦这些通过,就将它们适当地添加到回归套件中。您可能不需要回归套件中包含的每个边缘案例或排列,并且您希望保持回归套件足够快以提供及时的反馈。随着每个故事的完成,确认其功能的测试应该包含在回归套件中,并成为常规构建周期的一部分。

During the iteration, you’re automating new tests. As soon as these pass, add them to the regression suite, as appropriate. You may not need every edge case or permutation included in the regression suite, and you want to keep the regression suites fast enough to provide timely feedback. As each story is completed, tests that confirm its functionality should be included in the regression suite and be part of the regular build cycle.

回归测试本身必须处于某种形式的版本控制之下。最好将它们保存在与生产代码相同的源代码控制系统中。这样,当您为生产发布标记代码时,该标记还包含与该代码一起使用的所有测试版本。至少,每天备份测试代码。

The regression tests themselves must be under some form of version control. It’s best to keep them in the same source code control system as the production code. That way, when you tag the code for production release, the tag also contains all of the versions of the tests that worked with the code. At minimum, keep a daily backup of the test code.

当测试被添加到回归套件中时,它们的目的就会改变。它们不再存在以帮助推动开发,并且预计它们不会发现新的错误。生活中的唯一目的是检测系统中意想不到的变化或副作用。

When tests have been added to the regression suite, their purpose changes. They no longer exist to help drive development, and they are not expected to find new bugs. There sole purpose in life is to detect unexpected changes or side effects in the system.

检查“大局”

Checking the “Big Picture”

希望您编写了任务卡来在更大的应用程序的上下文中测试故事,并对系统的其他部分进行回归测试,以确保新故事没有产生负面影响。您可能已经自动化了其中一些端到端测试,例如第 12 章测试象限总结”中的示例。

Hopefully, you wrote task cards to test the story in the context of the larger application and regression test other parts of the system to ensure the new story hasn’t had a negative effect. You may have automated some of those end-to-end tests like the example in Chapter 12, “Summary of Testing Quadrants.”

但有时,即使您有一大套回归测试,手动探索性测试也是合适的。在您也完成这些任务之前,故事还没有“完成”。

But sometimes, even if you have a large suite of regression tests, manual exploratory testing can be appropriate. The story isn’t “done” until you’ve completed these tasks as well.

资源

Resources

当您开始迭代时,请确保测试环境、测试数据和测试工具已到位以适应测试此迭代的故事。希望您已经预料到这些需求,但有些需求可能只有在您开始编写故事时才会变得明显。与数据库专家、系统管理员和其他团队成员协作,以设置所需的任何其他基础设施。

As you start the iteration, make sure that test environments, test data, and test tools are in place to accommodate testing this iteration’s stories. Hopefully you’ve anticipated these needs, but some requirements might only become obvious when you start working on a story. Collaborate with database experts, system administrators, and other team members to set up any additional infrastructure needed.

您可能为此迭代引入了外部资源,以帮助进行性能、可用​​性、安全性或其他形式的测试。根据需要让他们参与与客户的站会和讨论。与他们配对并帮助他们了解团队的目标。这是学习新技能的机会。

You may have brought in outside resources for this iteration to help with performance, usability, security, or other forms of testing. Include them in stand-ups and discussions with the customers as needed. Pair with them and help them understand the team’s objectives. This is an opportunity to pick up new skills.

迭代指标

Iteration Metrics

第 5 章转换典型流程”中,我们略微谈到了度量的目的,但由于度量对于了解您的编码和测试活动的进展情况至关重要,因此我们将在此处更深入地研究它们。在开始测量数据点并开始分析结果的所有工作之前,了解您要解决的问题。在本节中,我们将介绍团队通过迭代收集的一些典型度量。

In Chapter 5, “Transitioning Typical Processes,” we talked a bit about the purpose of metrics, but because metrics are critical to understanding how your coding and testing activities are progressing, we’ll delve into them more here. Know what problem you are trying to solve before you start measuring data points and going to all the work of analyzing the results. In this section, we’ll cover some of the typical measurements that teams gather through the iteration.

第 15 章,“发布或主题规划中的测试人员活动”,讨论了要保留的有用指标。

Chapter 15, “Tester Activities in Release or Theme Planning,” talks about useful metrics to keep.

衡量进展

Measuring Progress

您需要一些方法来了解您的团队在迭代中的任何时间点完成了多少工作,以及还有多少工作要做。您需要知道何时某些故事无法完成并且团队需要 B 计划。迭代燃尽图和任务的估计时间与实际时间是用于衡量团队进度的示例。他们可能会也可能不会为您的特定团队提供价值。

You need some way to know how much work your team has completed at any point in the iteration and an idea of how much work is left to do. You need to know when it becomes obvious that some stories can’t be completed and the team needs a Plan B. Iteration burndown charts and estimated versus actual time for tasks are examples used to measure team progress. They may or may not provide value for your particular team.

故事板或任务板是了解迭代状态的良好视觉方式,尤其是在使用颜色编码的情况下。如果太多的测试任务卡仍然在“待办事项”栏中,或者没有足够的编码任务卡被移动到“完成”或“已测试”,那么团队是时候想办法确保所有测试都完成了完全的。也许一些团队成员需要停止编码并开始承担测试任务,或者可能需要将一个故事或故事中不太重要的部分推迟到下一次迭代,以便完成对所有其他故事的测试。

Story or task boards are a good visual way to know the iteration’s status, especially if color coding is used. If too many test task cards are still in the “to do” column or not enough coding task cards have been moved to “Done” or “Tested,” it’s time for the team to think of ways to make sure all of the testing is completed. Maybe some team members need to stop coding and start taking on testing tasks, or maybe one story or a less critical part of a story needs to be put off until the next iteration so that testing for all the other stories can be finished.

这可以通过虚拟故事板和物理故事板来实现。发挥视觉效果的创意,使问题立即可见。请记住,在所有适当的级别上测试之前,任何故事都不会“完成”。团队可能对故事何时“完成”有其他标准,例如它是否经过同行评审或自动回归测试是否完成。在图 18-6所示的故事板上,每个故事行的“完成”列是最右边的一列。它左边的列是“验证”列。在所有卡片(包括测试任务卡片)都位于“完成”列中之前,故事不会被视为“完成”。看一眼棋盘就足以知道哪些故事已经完成。

This can be accomplished with virtual story boards as well as physical ones. Get creative with your visual effects so that problems are instantly visible. Remember that no story is “done” until it’s tested at all appropriate levels. Teams may have other criteria for when a story is “done,” such as whether it has been peer reviewed or the automated regression tests are completed. On the story board shown in Figure 18-6, the “Done” column for each story row is the rightmost column. The column just to the left of it is the “Verify” column. The story isn’t considered “done” until all the cards, including testing task cards, are in that “Done” column. A glance at the board is enough to know which stories are finished.

图 18-6显示迭代故事和任务的故事板。经 Mike Thomas 许可使用。版权所有 2008。

Figure 18-6 Story board showing iteration stories and tasks. Used with permission of Mike Thomas. Copyright 2008.

图像

即使团队不在任务级别跟踪燃尽图,也可以在故事级别进行跟踪。了解团队每次迭代可以完成多少工作(其速度)有助于制定整体发布计划,并重新确定每次迭代的优先级。如果故事的大小趋于平均,那么知道在一次迭代中完成的故事数量就足够了。尽管计划充其量只是暂时的,但了解可以实现多少个故事是有帮助的在硬发布日期或下一季度可能完成的故事之前完成。

Even teams that don’t track burndown at the task level can do so at the story level. Knowing how much work the team can do each iteration (its velocity) helps with the overall release plan, and the reprioritizing for each iteration. It simply may be enough to know the number of stories completed in an iteration if they tend to average out to the same size. Although plans are tentative at best, it’s helpful to get an idea of about how many stories can be completed by a hard release date or what stories might get done in the upcoming quarter.

缺陷指标

Defect Metrics

我们在第 15 章发布或主题规划中的测试人员活动”中讨论了缺陷指标,为您提供了一些关于跟踪内容的高级概念。收集有关缺陷的指标可能非常耗时,因此在开始衡量之前始终考虑目标。您希望收集的指标的目的是什么?你需要跟风多久才能知道它们是否有用?

We talked about defect metrics in Chapter 15, “Tester Activities in Release or Theme Planning” giving you some high level ideas about what to track. Gathering metrics on defects can be very time consuming so always consider the goal before you start to measure. What is the purpose of the metrics you would like to gather? How long will you need to follow the trend before you know if they are useful?

缺陷控制始终是最喜欢捕获的指标。缺陷是什么时候发现的?在传统项目中,这要容易得多,因为您有“硬”要求和编码阶段。当整个团队都对质量负责,并且每个人都在自始至终一起工作时,就很难确定“何时”将缺陷注入系统。

Defect containment is always a favorite metric to capture. When was the defect found? In traditional projects, it is much easier as you have “hard” requirements and coding phases. When the whole team is responsible for quality, and everyone is working together throughout, it is much harder to determine “when” the defect was injected into the system.

我们想挑战这种类型的指标的想法,因为它在敏捷开发中不是必需的。但是,如果您发现有很多错误被忽略,您可能需要开始跟踪它们是什么类型的错误,以便解决根本原因。例如,如果错误可以通过单元测试发现,那么程序员可能需要更多关于编写单元测试的培训。如果错误被遗漏或误解了需求,那么可能没有足够的时间花在迭代计划上,或者验收测试不够详细。

We would like to challenge the idea of this type of metric as not necessary in agile development. However, if you find a lot of bugs are slipping through, you may want to start tracking what type of bugs they are so you can address the root cause. For example, if the bugs could have been caught with unit tests, then maybe the programmers need more training on writing unit tests. If the bugs are missed or misunderstood requirements, then maybe not enough time is spent in iteration planning, or acceptance tests aren’t detailed enough.

如果您对缺陷实行零容忍,那么您可能不需要在编码和测试期间跟踪缺陷。故事板上的一张简单卡片将为您提供所需的所有信息。

If you are practicing zero tolerance for defects, then you probably have no need to be tracking defects during coding and testing. A simple card on the story board will give you all the information you need.

无论您选择衡量什么指标,都要追求简单。

Whatever metrics you choose to measure, go for simplicity.

珍妮特的故事

Janet’s Story

在我所在的一个组织中,我们跟踪了多个版本中记录在 DTS 中的缺陷数量。这些是在迭代中逃脱或在遗留系统中发现的缺陷。图 18-7显示了一年半的趋势。

In one organization I was with, we tracked the number of defects logged in the DTS over several releases. These were defects that escaped the iteration or were found in the legacy system.Figure 18-7 shows the trend over a year and a half.

图 18-7样本缺陷趋势(但在一段时间后停止)

Figure 18-7 Sample Defect Trend (but stopped after a while)

图像

一开始,发布给QA做最终测试后发现的问题数量很高(一个月发现33个问题)。客户在持续两个多月的 UAT 期间发现了更多问题,因为他们没有对发布的质量充满信心。在报告零缺陷的那个月,我们刚刚开始发布新版本,因此没有要测试的新功能。在接下来的一年里,记录的缺陷越来越少,并且无法仅通过查看趋势来判断实际发布发生的位置。

At the beginning, the number of issues found right after it was released to QA for final testing was high (33 issues found in one month). The customers found even more issues during UAT which lasted over two months because they were not confident in the quality of the release. In the month that zero defects were reported, we were just starting a new release so there was no new functionality to test. Over the next year, fewer and fewer defects were logged and it becomes impossible to tell where an actual release happened by just looking at the trend.

该图用于向客户表明团队正在与他们的测试和发布保持一致。一旦团队和客户相信数字不会上升,指标就不再需要并被丢弃。

This graph was used to show customers that the team was becoming consistent with their testing and their releases. Once the team and customers had faith the numbers were not going up, the metrics were no longer needed and were dropped.

——珍妮特

—Janet

当指标不再有用时,不要害怕停止使用指标。如果他们最初收集的问题不再存在,就没有理由继续收集它们。

Don’t be afraid to stop using metrics when they are no longer useful. If the problem they were initially gathered for no longer exists, there is no reason to keep gathering them.

您的团队可能必须向高层管理人员或项目管理办公室 (PMO) 提供指标,尤其是当您为大型组织工作时。Patrick Fleisch,埃森哲顾问,曾在一家公司担任职能分析师在我们写这本书的时候,这家软件公司给了我们以下他的团队向他们的 PMO 提供的指标示例。

Your team may have to provide metrics to upper managers or a Project Management Office (PMO), especially if you work for a large organization. Patrick Fleisch, an Accenture Consultant who was working as a functional analyst at a software company during the time we wrote this book, gave us the following examples of metrics his team provides to their PMO.

图像按故事和功能区域测试执行数量

Test execution numbers by story and functional area

图像测试自动化状态(自动化测试与手动测试的数量)

Test automation status (number of tests automated vs. manual)

图像随着时间的推移通过/失败的测试数量的折线图

Line graph of the number of tests passing/failing over time

图像每个故事的摘要和状态

Summary and status of each story

图像缺陷指标

Defect metrics

收集和报告诸如此类的指标可能会导致大量开销。寻找最简单的方法来满足您的组织的需求。

Gathering and reporting metrics such as these may result in significant overhead. Look for the simplest ways to satisfy the needs of your organization.

概括

Summary

在我们的示例迭代的这一点上,我们的敏捷测试人员与程序员、客户和其他团队成员密切合作,以小的测试-编码-审查-测试增量生成故事。要记住的几点是:

At this point in our example iteration, our agile tester works closely with programmers, customers, and other team members to produce stories in small testing-coding-reviewing-testing increments. Some points to keep in mind are:

图像编码和测试是迭代过程中的一个过程的一部分。

Coding and testing are part of one process during the iteration.

图像编码一开始就为故事编写详细的测试。

Write detailed tests for a story as soon as coding begins.

图像从简单的测试开始推动开发;当简单测试通过后,编写更复杂的测试用例以进一步指导编码。

Drive development by starting with a simple test; when the simple tests pass, write more complex test cases to further guide coding.

图像使用简单的风险评估技术来帮助集中测试工作。

Use simple risk assessment techniques to help focus testing efforts.

图像当要求不明确或意见不同时,使用“三的幂”。

Use the “Power of Three” when requirements aren’t clear or opinions vary.

图像专注于一次完成一个故事。

Focus on completing one story at a time.

图像与程序员密切合作,以便集成测试和编码。

Collaborate closely with programmers so that testing and coding are integrated.

图像批评产品的测试是开发的一部分。

Tests that critique the product are part of development.

图像在整个迭代过程中让客户了解情况;让他们早点经常复习。

Keep customers in the loop throughout the iteration; let them review early and often.

图像团队中的每个人都可以从事测试任务。

Everyone on the team can work on testing tasks.

图像 测试人员可以促进客户团队和开发团队之间的沟通。

Testers can facilitate communication between the customer team and development team.

图像确定对你的团队来说最好的“错误修复”选择是什么,但一个好的目标是在发布时没有错误。

Determine what the best “bug fixing” choice for your team is, but a good goal is to aim to have no bugs by release time.

图像将新的自动化测试添加到回归套件中,并安排它经常运行以提供足够的反馈。

Add new automated tests to the regression suite and schedule it to run often enough to provide adequate feedback.

图像在对所有应用程序进行编码后,手动探索性测试有助于找到缺失的需求。

Manual exploratory testing helps find missing requirements after all the application has been coded.

图像与其他专家合作以获得完成测试所需的资源和基础设施。

Collaborate with other experts to get the resources and infrastructure needed to complete testing.

图像考虑迭代期间需要哪些指标;进度和缺陷指标是两个例子。

Consider what metrics you need during the iteration; progress and defect metrics are two examples.

第 19 章 结束迭代

Chapter 19 Wrap Up the Iteration

我们已经完成了一次迭代。当团队结束本次迭代并为下一次做准备时,测试人员会做什么?我们喜欢专注于我们和我们团队的其他成员下次如何改进和交付更好的产品。

We’ve completed an iteration. What do testers do as the team wraps up this iteration and prepares for the next? We like to focus on how we and the rest of our team can improve and deliver a better product next time.

迭代演示

Iteration Demo

敏捷开发的乐趣之一是有机会在每次迭代结束时向客户展示已完成的故事。客户可以看到真实、实时、工作的应用程序。他们可以提出问题并提供反馈。参与该项目的每个人,无论是业务方面还是技术方面,都会获得成就感。

One of the pleasures of agile development is the chance to show completed stories to customers at the end of each iteration. Customers get to see a real, live, working application. They get to ask questions and give feedback. Everyone involved in the project, from both the business and technical sides, gets to enjoy a sense of accomplishment.

在 Lisa 的团队中,测试人员进行迭代评审。在所有团队成员中,他们通常处理的故事最多。他们天生就是信息提供者,他们很清楚客户需要了解新功能的哪些方面。让测试人员展示可交付成果是一种常见的做法,尽管没有硬性规定。团队中的业务专家也是进行演示的不错选择,因为他们最了解软件如何满足业务需求,并且他们会对产品有更大的主人翁感。ScrumMaster、程序员或业务分析师可以演示新功能并且经常这样做。珍妮特鼓励轮换这一荣誉。

On Lisa’s team, the testers conduct the iteration review. Among all the team members, they’ve usually worked on the most stories. They have a natural role as information providers, and they have a good idea what the customers need to know about the new functionality. Having testers show off the deliverables is a common practice, although there is no hard and fast rule. The business experts on the team are a good choice for conducting the demo too, because they have the best understanding of how the software meets the business needs and they’ll feel greater ownership of the product. The ScrumMaster, a programmer, or a business analyst could demonstrate the new features and often does. Janet encourages rotating this honor.

任何人都可能注意到客户在参与演示时发表的评论,但测试人员是不错的人选。随着演示的进行,他们可能会注意到以前未检测到的不一致。随着问题的出现,客户可能会决定他们想要改变一些小的东西,比如帮助文本,或者更大的东西,比如一个特性的行为方式。微小的变化通常可以变成任务并在下一次迭代中处理,但有些变化大到足以变成故事来计划未来的发布。

Anyone may note the comments made by customers as they participate in the demo, but testers are good candidates. They may notice previously undetected inconsistencies as the demo progresses. As questions come up, customers might decide they want to change something minor, such as help text, or something bigger, such as how a feature behaves. Minor changes can usually be made into tasks and dealt with in the next iteration, but some changes are big enough to turn into stories to plan into future releases.

迭代演示(在 Scrum 世界中称为冲刺评审)是让每个人讨论和思考应用程序的绝佳机会。利用它。审查会议通常很短,可能不到半小时。如果在展示新故事后还有剩余时间,请询问客户是否遇到过他们未报告的先前版本的任何问题。他们是否有任何一般性顾虑,是否需要帮助以了解如何使用某项功能,或者是否出现了任何新问题?当然,您可以随时与客户交谈,但是让大多数利益相关者与开发团队同在一个房间可以产生有趣的想法。

Iteration demos (called sprint reviews in the Scrum world) are a super opportunity to get everyone talking and thinking about the application. Take advantage of it. Review meetings are usually short and can be under half an hour. If there’s time left over after demonstrating new stories, ask customers if they’ve experienced any problems with the previous release that they haven’t reported. Do they have any general concerns, do they need help understanding how to use a feature, or have any new issues arisen? Of course, you can talk to customers anytime, but having most of the stakeholders in the room with the development team can lead to interesting ideas.

回顾

Retrospectives

敏捷开发意味着不断改进您的工作方式,而回顾是开始确定您可以做得更好的地方以及如何做得更好的好地方。我们建议在每次迭代和发布周期结束时花点时间回顾并讨论哪些进展顺利,哪些进展不顺利,以及您可能希望在下一次迭代中尝试哪些内容。进行回顾会议有不同的方法。无论您使用哪种方法,关键是每个团队成员都感到安全,每个人都受到尊重,并且没有指责或责备。

Agile development means continually improving the way you work, and retrospectives are an excellent place to start identifying what and how you can do better. We recommend taking time at the end of each iteration and release cycle to look back and talk about what went well, what didn’t, and what you might like to try in the next iteration. There are different approaches for conducting retrospective sessions. No matter what approach you use, it’s key that each team member feels safe, everyone is respected, and there’s no finger-pointing or blame.

整个想法是让这个过程变得更好,一次一小步。

The whole idea is to make the process better, one baby step at a time.

开始、停止、继续

Start, Stop, Continue

迭代回顾中使用的一种常见练习是“开始、停止、继续”。团队自问:“在过去的迭代中,哪些进展顺利?发生了不该再发生的事?我们可以开始做什么来帮助处理不顺利的事情?” 每个团队成员都可以建议开始做些什么来改进,停止做没有效果的事情,以及应该继续做的有帮助的事情。协调员或 ScrumMaster 将它们列在白板或大纸上。将它们张贴在每个人都可以在迭代期间再次阅读的位置。图 19-1显示了正在进行的“停止、开始和继续”回顾。ScrumMaster(站立)正在故事板上的大纸上写下停止、开始和继续建议。

One common exercise used in iteration retrospectives is “start, stop, continue.” The team asks itself: “What went well during this past iteration? What happened that shouldn’t happen again? What can we start doing to help with things that didn’t go well?” Each team member can suggest things to start doing to improve, things to stop doing that weren’t working, and things that are helping that should be continued. A facilitator or ScrumMaster lists them on a whiteboard or big piece of paper. Post them in a location where everyone can read them again during the iteration. Figure 19-1 shows a “stop, start, and continue” retrospective in progress. The ScrumMaster (standing) is writing stop, start, and continue suggestions on the big piece of paper on the story board.

图 19-1正在进行的回顾。经 Mike Thomas 许可使用。版权所有 2008。

Figure 19-1 A retrospective in progress. Used with permission of Mike Thomas. Copyright 2008.

图像

Agile Retrospectives: Making Good Teams Great [2006] 具有使回顾更有成效的富有想象力的想法(参见参考书目

Agile Retrospectives: Making Good Teams Great [2006] has imaginative ideas for making retrospectives more productive (see the bibliography).

一些团队提前开始这个过程。所有团队成员在便利贴上写下“开始”、“停止”和“继续”项目,然后在回顾会议期间,他们将便利贴贴在黑板上并按主题分组。“开始、停止、继续”只是您可能使用的术语之一。其他一些想法是:“进展顺利的事情”、“需要改进的事情”、“愉快的事情”、“令人沮丧的事情”和“尝试一下”。使用适合您的任何名称。有可能很难记住过去两周,更不用说整个版本了,如果那是你的回顾涵盖的内容。研究不同的创造性方法来反映您团队的经验。

Some teams start this process ahead of time. All team members write “start,” “stop,” and “continue” items on sticky notes, and then during the retrospective meeting they put the stickies on the board and group them by topic. “Start, stop, continue” is just one example of the terms you might use. Some other ideas are: “Things that went well,” “Things to improve,” “Enjoyable,” “Frustrating,” and “To Try.” Use whatever names that work for you. It can be hard to remember the past two weeks, much less an entire release, if that’s what your retrospective covers. Research different creative approaches to reflecting on your team’s experiences.

这是 Lisa 团队的“停止、开始、继续”列表示例:

Here’s a sample “stop, start, continue” list from Lisa’s team:

开始:

Start:

图像提前向我们发送下一个 sprint 的故事。

Sending out next sprint’s stories to us earlier.

图像不要进行懒惰的单记录处理。将每个服务调用都视为远程调用。

Don’t do lazy, single-record processing. Think of every service call as a remote call.

图像将任何数据库更改传达给每个人。

Communicate any database changes to everyone.

停止:

Stop:

图像接受没有完整要求的故事。

Accepting stories without complete requirements.

继续:

Continue:

图像为您正在处理的代码运行 FitNesse 测试。

Running FitNesse tests for the code you’re working on.

图像记录会议或非正式讨论中出现的内容。

Documenting what came up in meeting or informal discussions.

图像彼此更好地沟通。

Communicating better with each other.

图像尽早展示模型。

Showing mock-ups early.

图像进行 FitNesse 驱动的开发。

Doing FitNesse driven development.

如果“开始、停止、继续”项目列表很长,最好选择一两个项目作为新迭代的重点。为了确定项目的优先级,给每个团队成员“n”票,他们可以分配给项目。Lisa 团队中的十个人每人获得三票,如果他们认为最重要,他们可以将所有票投给一个项目,或者他们可以投票给两个或三个不同的项目。得票最多的项目被标记为焦点项目。珍妮特也通过这种确定优先次序的方式取得了成功。

If the list of “start, stop, continue” items is long, it’s a good idea to choose one or two to focus on for the new iteration. To prioritize the items, give each team member “n” votes they can assign to items. The ten people on Lisa’s team each get three votes, and they can apply them all to one item if they feel that’s most important, or they can vote for two or three different items. The items with the most votes are noted as the focus items. Janet has had success with this way of prioritizing as well.

除了“开始、停止、继续”项目之外,团队还可以简单地为下一次迭代要采取的行动编写任务卡。例如,如果正在进行的构建速度太慢,请写一张卡片“在十分钟内完成正在进行的构建”。

In addition to “start, stop, continue” items, the team may simply write task cards for actions to be undertaken the next iteration. For example, if the ongoing build is too slow, write a card to “get ongoing build under ten minutes.”

在下一次迭代中,花一些时间查看您想要改进的一两个重点项目。在那次迭代结束时,做一个检查点看看你是否有所改进。如果不是,请问为什么。你应该尝试不同的东西吗?它还重要吗?可能是它的重要性下降了,或者从大局来看真的不重要。如果你认为你在某个问题领域有所改进,但它又重新浮出水面,你将不得不决定对此做些什么,否则就不再谈论它。

In the next iteration, take some time to look at the one or two focus items you wanted to improve. At the end of that iteration, take a checkpoint to see if you improved. If not, ask why. Should you try something different? Is it still important? It could be it has dropped in importance or really wasn’t important in the big picture. If you thought you improved on a problem area and it resurfaces, you’ll have to decide to do something about it or else quit talking about it.

我们发现回顾是团队识别和解决问题的一种简单而高效的方法。回顾会议是提出测试相关问题的绝好机会。以客观、非责备的方式提出问题。团队可以讨论每个问题,可能导致问题的原因,并写下一些解决问题的想法。

We’ve found that retrospectives are a simple and highly effective way for teams to identify and address issues. The retrospective meeting is a perfect opportunity to raise testing-related issues. Bring up the issues in an objective, non-blaming way. The team can discuss each problem, what might be causing it, and write down some ideas to fix it.

改进的想法

Ideas for Improvements

让我们来看看其中一些被列入改进清单的项目。很多时候,一个团队会发现非常大的问题,但从不跟进并实际采取措施解决它们。例如,在程序员声称编码完成后,可能会发现很多单元级错误。

Let’s take a look at some of those items that made it onto the list for improvement. Too many times, a team will identify really big issues but never follow up and actually do something about them. For example, maybe a lot of unit-level bugs are discovered after the programmers have claimed coding was complete.

团队可能会认为程序员没有用单元测试覆盖足够的代码。他们可能会在签入新代码之前编写一个操作项来运行代码覆盖工具,或者开始为每个故事编写“单元测试”任务卡以确保它们已完成。也许团队在迭代结束之前没有完成所有的测试自动化任务。当他们讨论这个问题时,团队发现最初的可执行测试过于复杂,他们需要先专注于编写和自动化一个简单的测试,或者配对以获得更好的测试设计。确保行动项目是具体的。

The team may decide the programmers aren’t covering enough code with unit tests. They might write an action item to run the code coverage tool before they check in new code, or start writing a “unit tests” task card for each story to make sure they’re completed. Perhaps the team didn’t finish all the test automation tasks before the iteration ended. As they discuss the problem, the team finds that the initial executable tests were too complex, and they need to focus on writing and automating a simple test first, or pair for a better test design. Make sure the action items are concrete.

敏捷团队尝试解决他们自己的问题并制定指导方针来帮助他们自己改进。针对一个问题的行动项目可能对其他问题有帮助。当 Lisa 的团队在完成故事并在每次迭代中对其进行测试时遇到困难时,他们在几次回顾的过程中提出了各种规则:

Agile teams try to solve their own problems and set guidelines to help themselves improve. Action items aimed at one problem may help with others. When Lisa’s team had trouble finishing stories and getting them tested during each iteration, it came up with various rules over the course of a few retrospectives:

图像在迭代的第四天之前完成所有故事的高级测试用例。

Finish high-level test cases for all stories by the fourth day of the iteration.

图像在迭代的第四天交付一个故事进行测试。

Deliver one story to test by the fourth day of the iteration.

图像专注于一次完成一个故事。

Focus on finishing one story at a time.

图像100% 的功能必须在迭代的倒数第二天下班前签入。

100% of features must be checked in by close of business on the next-to-last day of the iteration.

这些规则不仅仅帮助团队完成了测试任务。他们促进了流程和节奏,帮助团队在每次迭代过程中以稳定、可持续的速度工作。

These rules did more than help the team finish testing tasks. They facilitated a flow and rhythm that helped the team work at a steady, sustainable pace over the course of each iteration.

通过回顾行动项目来开始下一次回顾会议,看看哪些项目是有益的。丽莎的团队将快乐、悲伤或中性的面孔放在旁边项目来表示团队是否尝试过它们并发现它们是成功的。团队应该找出任何悲伤面孔背后的原因。是不是有些项目被遗忘了?时间限制是否阻止了团队尝试新活动?只是后来似乎不是一个好主意吗?这些讨论可能会导致更改改进项目或将其演化为新项目。

Begin the next retrospective meeting by reviewing the action items to see what items were beneficial. Lisa’s team puts happy, sad, or neutral faces next to items to denote whether the team tried them and found them successful. The team should figure out the reasons behind any sad faces. Were some items simply forgotten? Did time constraints keep the team from trying a new activity? Did it just seem to be less of a good idea later? These discussions might lead to changing the improvement item or evolving it into a new one.

当改进行动成为团队的习惯时,它们就不再需要写在“停止、开始和继续”列表上。运行良好的“开始”项目可能会移至“继续”列。有些想法行不通,或者被证明是不必要的,这些也可以从下一次迭代的列表中删除。

When the actions for improvement become a habit to the team, they no longer need to be written on the “stop, start, and continue” list. “Start” items that work well may be moved to the “Continue” column. Some ideas don’t work, or prove to be unnecessary, and those can also be taken off the list for the next iteration.

在迭代期间参考您的改进想法和行动项目。将它们张贴在每个人都经常看到的位置(墙上或网上)。Lisa 的团队有时会在迭代中期的站立会议中仔细检查列表。如果您在迭代过程中想到新的改进想法,请将它们写下来,甚至可能在现有列表中,这样您就不会在下一次迭代时忘记。

Refer to your ideas for improvement and action items during the iteration. Post them in a location (on a wall or online) where everyone sees them often. Lisa’s team sometimes goes through the list during a mid-iteration stand-up meeting. If you think of new improvement ideas during the iteration, write them down, possibly even on the existing list, so you won’t forget for the next iteration.

在整个迭代过程中跟踪阻碍您前进的事情是个好主意。在一些大的可见图表上保留障碍积压。讨论每次迭代中的障碍,写任务卡或采取行动消除它们。

It’s a good idea to keep track of things that get in your way throughout the iteration. Keep an impediment backlog on some big visible chart. Talk about the impediments in each iteration, and write task cards or take action to eliminate them.

使用回顾作为提出与测试相关的问题并让整个团队思考可能的解决方案的机会。我们对整个团队提出的创新想法感到惊喜,他们专注于如何改进其工作方式。

Use retrospectives as an opportunity to raise testing-related issues and get the whole team thinking about possible solutions. We’ve been pleasantly surprised with the innovative ideas that come out of an entire team focusing on how to improve the way it works.

庆祝成功

Celebrate Successes

敏捷开发实践倾向于缓和更传统或混乱过程中存在的高潮和低谷。如果您的瀑布团队在长达一年的周期以两个月的紧张修复和测试周期结束后最终设法推出了一个版本,那么每个人都可能准备好举办一个盛大的派对来庆祝这一事件——或者他们可能只是崩溃了几个星期。每两周发布一次的敏捷团队倾向于保持正常的编码和测试状态,在吸取足够的呼吸以进行迭代审查和回顾后开始下一组故事。这很好,但你知道他们怎么说只工作不玩耍。

Agile development practices tend to moderate the highs and lows that exist in more traditional or chaotic processes. If your waterfall team finally manages to push a release out the door after a year-long cycle ending in a two-month stressful fix-and-test cycle, everyone may be ready to celebrate the event with a big party—or they might just collapse for a couple of weeks. Agile teams that release every two weeks tend to stay in their normal coding and testing groove, starting on the next set of stories after drawing just enough breath to hold an iteration review and retrospective. This is nice, but you know what they say about all work and no play.

确保你的团队至少花一点时间来表扬自己并认可他们的成就。即使是很小的成功也应该得到奖励。享受是一种至关重要的敏捷价值,一点点动力可以帮助您的团队在成功的道路上继续前进。出于某种原因,这可能很难做到。许多敏捷团队很难花时间庆祝成功。有时您急于开始下一次迭代,而没有花时间为之前的成就祝贺自己。

Make sure your team takes at least a little time to pat itself on the back and recognize its achievements. Even small successes deserve a reward. Enjoyment is a vital agile value, and a little motivation helps your team continue on its successful path. For some reason, this can be hard to do. Many agile teams have trouble taking time to celebrate success. Sometimes you’re eager to get going with the next iteration and don’t take time to congratulate yourselves on the previous accomplishments.

Lisa 的团队每隔一个星期四结束一次迭代,并在第二天进行回顾、迭代审查和发布。会议结束后,他们通常会进行他们称之为“周五娱乐”的活动。这有时包括玩一些愚蠢的琐事或棋盘游戏、出去喝一杯或打一场迷你高尔夫球。有机会放松和大笑有利于团队建设。

Lisa’s team ends an iteration every other Thursday and conducts its retrospective, iteration review, and release the following day. After their meetings conclude, they usually engage in something they call “Friday Fun.” This sometimes consists of playing a silly trivia or board game, going out for a drink, or playing a round of miniature golf. Getting a chance to relax and have a good laugh has a team-building side benefit.

对于更大的里程碑,例如重大发布或实现测试覆盖率目标,整个公司都会举行派对庆祝,带来餐饮或外出星期五下午去一家餐馆。这是对业务和技术团队中每个人的一个很好的奖励和认可。

For bigger milestones, such as a big release or achieving a test coverage goal, the whole company has a party to celebrate, bringing in catered food or going out to a restaurant on Friday afternoon. This is a nice reward and recognizes for everyone on both the business and technical teams.

如果您是一个新的敏捷团队,请通过奖励小成就来激励自己。为每个构建中通过的单元测试数量增加而欢呼。图表显示实际燃烧量与预计燃烧量相匹配。当构建中损坏的单元测试被修复时按铃(好吧,那个可能很烦人,但以某种方式识别它。)

If yours is a new agile team, motivate yourselves by rewarding small accomplishments. Cheer the rising number of unit tests passing in each build. Oooh and aaah over the chart showing actual burn down matching the projected burn down. Ring a bell when the broken unit tests in the build are fixed (okay, that one might be annoying, but recognize it in some way.)

也要庆祝你个人的成功。祝贺您的同事完成了项目的第一个性能测试基线。给你的 DBA 一个金星来实现生产备份系统。奖励自己解决那个困难的测试自动化问题。下次与客户会面时带上饼干。认识为您提供 JavaScript 工具以加速某些 GUI 验证测试的程序员。动用你的想象力。

Celebrate your individual successes, too. Congratulate your coworker for completing the project’s first performance test baseline. Give your DBA a gold star for implementing a production back-up system. Give yourself a treat for solving that hard test-automation problem. Bring cookies to your next meeting with the customers. Recognize the programmer who gave you a JavaScript harness that sped up testing of some GUI validations. Use your imagination.

花时间庆祝成功可以让您的团队后退一步,获得全新的视角,并重新焕发活力,从而不断改进您的产品。让团队成员有机会欣赏彼此的贡献。不要陷入每个人都一直低着头工作的例行公事。

Taking time to celebrate successes lets your team take a step back, get a fresh perspective, and renew its energy so it can keep improving your product. Give team members a chance to appreciate each other’s contributions. Don’t fall into a routine where everyone has their head down working all the time.

在敏捷开发中,我们有机会在每次短迭代结束时停下来并获得新的视角。我们可以进行细微的课程修正,决定尝试一种新的测试工具,想出更好的方法来从客户那里引出示例,或者确定对特定类型测试专业知识的需求。

In agile development, we get a chance to stop and get a new perspective at the end of each short iteration. We can make minor course corrections, decide to try out a new test tool, think of better ways to elicit examples from customers, or identify the need for a particular type of testing expertise.

概括

Summary

图像在本章中,我们研究了一些结束迭代或发布的活动。

In this chapter, we looked at some activities to wrap up the iteration or release.

图像迭代评审是从客户团队获得反馈和输入的绝佳机会。

The iteration review is an excellent opportunity to get feedback and input from the customer team.

图像回顾是帮助您的团队改进的重要实践。

Retrospectives are a critical practice to help your team improve.

图像查看团队可以改进的所有领域,但一次只关注一两个领域。

Look at all areas where the team can improve, but focus on one or two at a time.

图像找到一种在迭代过程中牢记改进项的方法。

Find a way to keep improvement items in mind during the iteration.

图像庆祝大大小小的成功,并认可不同角色和活动的贡献。

Celebrate both big and small successes, and recognize the contributions from different roles and activities.

图像利用每次迭代后的机会来识别与测试相关的障碍,并想办法克服它们。

Take advantage of the opportunity after each iteration to identify testing-related obstacles, and think of ways to overcome them.

第20章成功投递

Chapter 20 Successful Delivery

图像

在本章中,我们将分享您作为测试人员可以做些什么来帮助您的团队和您的组织成功交付高质量的产品。相同的流程和工具可用于收缩包装产品、定制解决方案或内部开发产品。敏捷测试人员可以做出独特的贡献,帮助客户和开发团队定义和产生业务所需的价值。

In this chapter, we share what you as a tester can do to help your team and your organization successfully deliver a high-quality product. The same process and tools can be used for shrink-wrapped products, customized solutions, or internal development products. Agile testers can make unique contributions that help both the customer and developer team define and produce the value that the business needs.

是什么造就了产品?

What Makes a Product?

许多关于敏捷开发的书籍都在谈论实际的开发周期,但忽略了如何制造产品以及成功交付该产品需要什么。仅仅编写代码、测试并说出它是不够的完毕。这就像从商店买东西:如果购买时有很好的服务,您再去那里购买的可能性有多大?

Many of the books on agile development talk about the actual development cycle but neglect to talk about what makes a product and what it takes to successfully deliver that product. It’s not enough to just code, test, and say it’s done. It’s like buying something from a store: If there is great service to go with the purchase, how much more likely are you to go back and buy there again?

珍妮特的故事

Janet’s Story

我正在和我的朋友罗恩聊天,他买卖硬币。多年来,他在业内建立了非常好的声誉,并且因为太忙而拒绝了潜在客户。

I was talking to my friend, Ron, who buys and sells coins. Over the years he has developed a very good reputation in the industry and has turned away prospective clients because he is so busy.

当我问起他的秘密时,他说:“这不是秘密。我只是与我的客户一起工作,让他们感到舒服,并与他们建立信任关系。最后,我和我的客户都需要对这笔交易感到满意。只要有一个不满意的顾客就会毁了我的名声。”

When I asked him his secret, he said, “It’s not a secret. I just work with my customers to make them feel comfortable and establish a trusting relationship with them. In the end, both I and my customer need to be happy with the deal. It only takes one unhappy customer to break my reputation.”

敏捷团队可以从 Ron 的经验中学习。如果我们尊重我们的客户并提供他们满意的产品,我们将与他们保持良好的关系,并有望持续多年。

Agile teams can learn from Ron’s experience. If we treat our customers with respect and deliver a product they are happy with, we will have a good relationship with them, hopefully for many years.

——珍妮特

—Janet

我们的目标是及时为企业创造价值。我们不仅要满足要求,还要取悦我们的客户。在我们发布之前,我们要确保所有可交付成果都已准备就绪并得到适当完善。希望您尽早开始计划,不仅要满足代码要求,还要计划培训、文档以及制作高价值产品所需的一切。

Our goal is to deliver value to the business in a timely manner. We don’t want just to meet requirements but also to delight our customers. Before we release, we want to make sure all of the deliverables are ready and polished up appropriately. Hopefully, you started planning early to meet not only the code requirements but to plan for training, documentation, and everything that goes into making a high-value product.

想象一下您正在为您的发布做好生产准备。您刚刚完成了最后一次迭代并结束了最后一个故事测试。您的自动回归套件一直在每个新构建上运行,或者至少在每个夜间构建上运行。你现在做什么将取决于你的过程有多严格。如果您一直对错误保持“零容忍”,那么您的状态可能非常好。

Imagine yourself in the middle of getting your release ready for production. You’ve just finished your last iteration and are wrapping up your last story test. Your automated regression suite has been running on every new build, or at least on every nightly build. What you do now will depend on how disciplined your process has been. If you’ve been keeping to the “zero tolerance” for bugs, you’re probably in pretty good shape.

如果您是那些认为可以将错误留到最后修复的团队之一,那么您的状态可能不太好,可能需要引入迭代以“强化”或修复错误。我们不建议这样做,但如果您的团队在开发周期中引入了很多突出的错误,则您需要在进入最终游戏之前解决这些错误。我们发现新团队往往会陷入这个陷阱。

If you’re one of those teams that thinks you can leave bugs until the end to fix, you’re probably not in such good shape and may need to introduce an iteration for “hardening” or bug fixes. We don’t recommend this, but if your team has a lot of outstanding bugs that have been introduced during the development cycle, you need to get those addressed before you go into the end game. We find that new teams tend to fall into this trap.

此外,任何版本都有许多不同的组件,一些在软件中,一些不在软件中。您的客户需要安装和学习使用新功能。考虑所有对成功发布至关重要的元素,因为是时候结束所有这些未解决的问题并磨练您的产品了。

In addition, there are lots of varied components to any release, some in the software, some not. You have customers who need to install and learn to use the new features. Think about all those elements that are critical to a successful release, because it’s time to wrap up all those loose ends and hone your product.

Bob Galen 是一名敏捷教练和终局专家,他观察到敏捷开发可能并没有渗透到组织的每个角落。他指出,“在软件的物理交付方面,敏捷测试人员可以充当管道或促进者。”

Bob Galen, an agile coach and end-game expert, observes that agile development may not have seeped into every organizational nook and cranny. He notes, “Agile testers can serve as a conduit or facilitator when it comes to physical delivery of the software.”

计划足够的时间进行测试

Planning Enough Time for Testing

因为测试和编码是敏捷开发中一个过程的一部分,所以我们不希望为额外的测试时间制定特殊的计划,但在现实生活中我们可能需要一些额外的时间。

Because testing and coding are part of one process in agile development, we’d prefer not to make special plans for extra testing time, but in real life we might need some extra time.

大多数团队都会积累一些技术债务,尽管本意是好的,尤其是当他们使用遗留代码时。为了保持速度,您的团队可能需要定期计划重构迭代以添加测试、升级工具和减少技术债务。Lisa 的团队大约每六个月进行一次重构冲刺。虽然业务通常不会在重构冲刺结束时获得任何直接收益,但业务专家明白这些特殊冲刺会带来更好的测试覆盖率、为未来开发打下坚实的基础、减少技术债务以及更高的整体团队速度.

Most teams accumulate some technical debt, despite the best intentions, especially if they’re working with legacy code. To maintain velocity, your team may need to plan a refactoring iteration at regular intervals to add tests, upgrade tools, and reduce technical debt. Lisa’s team conducts a refactoring sprint about every six months. While the business doesn’t usually receive any direct benefits at the end of a refactoring sprint, the business experts understand that these special sprints result in better test coverage, a solid base for future development, reduced technical debt, and a higher overall team velocity.

一些团队求助于“强化”迭代,他们只花时间查找和修复错误,而不引入任何新功能。这是保持应用程序及其基础架构稳固的最后手段。新团队可能需要额外的迭代来完成测试任务,如果是这样,他们会在发布计划中为此预算时间。

Some teams resort to “hardening” iterations, where they spend time only finding and fixing bugs, and they don’t introduce any new functionality. This is a last resort for keeping the application and its infrastructure solid. New teams may need an extra iteration to complete testing tasks, and if so, they budget time for that in the release plan.

使用回顾和其他流程改进实践来学习集成测试和编码的方法,以便每次迭代中生成的代码都可以投入生产。实现该目标后,努力确保每天都有可以发布到生产环境的稳定构建。Lisa 的团队成员认为,在他们努力在发布前获得任何稳定版本的日子里,这是一个无法实现的目标,但仅仅过了几年,几乎每个版本都值得发布。

Use retrospectives and other process improvement practices to learn ways to integrate testing and coding so that the code produced in each iteration is production-ready. When that goal is achieved, work to ensure that a stable build that could be released to production is available every day. Lisa’s team members thought that this was an unattainable goal in the days when they struggled to get any stable build before release, but it was only a couple of years before almost every build was release-worthy.

当您的构建稳定后,您就可以进入“终结游戏”了。

When your build is stable, you are ready to enter the “End Game.”

结束游戏

The End Game

什么是终极游戏?我们听说人们在交付很多东西之前就把时间称为交付时间,但“结束游戏”似乎最合适。这是团队对产品进行最后润色的时候。你正在点你的 i 并穿过你的 t。这是交付终点线前的最后一段路段。这并不是一个错误修复周期,因为到那时你不应该有任何未解决的错误,但这并不意味着你可能没有一两个要修复的。

What is the end game? We’ve heard people call the time right before delivery many things, but the “end game” seems to fit best. It’s the time when the team applies the finishing touches to the product. You’re dotting your i’s and crossing your t’s. It’s the last stretch before the delivery finish line. It’s not meant to be a bug-fix cycle, because you shouldn’t have any outstanding bugs by then, but that doesn’t mean you might not have one or two to fix.

您的组织中可能有您没有参与早期计划的小组。现在是时候与管理暂存和生产环境的人员、配置经理、团队外的数据库管理员以及在将软件从开发转移到暂存和生产过程中发挥作用的每个人密切合作了。如果这次您没有尽早与他们合作,请考虑在下一次发布计划会议期间与他们交谈,并在整个开发周期中与他们保持联系。

You might have groups in your organization that you didn’t involve in your earlier planning. Now it’s time to work closely with the folks that administer the staging and production environments, the configuration managers, the database administrators outside of your team, and everyone who plays a role in moving the software from development to staging and production. If you weren’t working with them early this time, consider talking to these folks during your next release planning sessions, and keep in touch with them throughout the development cycle.

Bob Galen 告诉我们,他团队中的测试人员与管理暂存和生产环境的运营团队合作。由于运营团队位于远程,因此发现敏捷团队的指导特别有价值。

Bob Galen tells us that the testers on his team have partnered with the operations group that manages the staging and production environments. Because the operations group is remote, it finds that having guidance from the agile team is particularly valuable.

总有一些系统级测试无法自动化,或者不值得自动化。通常情况下,您的暂存环境是唯一的地方您可以在其中进行一些系统级集成测试或系统级负载和压力测试。我们建议您在开发完成后分配一些时间来完成这些类型的完成任务。不要一直写到最后。

There are always system-level tests that can’t be automated, or are not worth automating. More often than not, your staging environment is the only place where you can do some system-level integration tests or system-level load and stress testing. We suggest that you allot some time after development for these types of finishing tasks. Don’t code right up to the end.

根据需要为结束游戏计划尽可能多的时间。Janet 发现,结束游戏所需的时间长短随团队的成熟度和应用程序的大小而变化。可能只需要一天就可以完成额外的任务,但也可能需要一周,有时甚至需要整整两周的迭代。第 12 章测试象限总结”中使用的示例中的团队安排了两周时间,因为这是一个复杂的系统,需要相当多的设置和系统测试。

Plan as much time for the end game as you need. Janet has found that the length of time needed for the end game varies with the maturity of the team and the size of the application. It may be that only one day is needed to finish the extra tasks, but it may be one week or sometimes as much as a whole two-week iteration. The team from the example used in Chapter 12, “Summary of Testing Quadrants,” scheduled two weeks, because it was a complex system that required a fair bit of setup and system testing.

丽莎的故事

Lisa’s Story

当我在一个为客户开发应用程序的团队工作时,我们必须遵循客户的发布时间表。只能在某些两周的窗口内,每六或八周,对更大系统的其他部分进行测试。我们的团队完成了两到三个迭代,完成了每个故事的所有故事,就好像他们在发布每个迭代一样。

When I worked on a team developing applications for a client, we had to follow the client’s release schedule. Testing with other parts of the larger system was only possible during certain two-week windows, every six or eight weeks. Our team completed two or three iterations, finishing all of the stories for each as if they were releasing each iteration.

然后我们进入了一个测试窗口,在这里我们可以与其他开发团队协调系统测试,协助客户进行 UAT,并计划实际发布。这构成了我们的最终游戏。

Then we entered a testing window where we could coordinate system testing with other development teams, assist the client with UAT, and plan the actual release. This constituted our end game.

——丽莎

—Lisa

如果您有一个大型组织,您可能有十个或十五个团队为单个产品或同一应用程序的不同功能区域开发软件。这些领域或者产品可能都需要一起发布,所以一个整合的端游是很有必要的。这并不意味着您将集成留到最后。在您的整个开发周期中,与其他团队的协调至关重要,如果您有测试集成系统,我们建议您确保在游戏结束之前很久就尝试过集成。

If you have a large organization, you might have ten or fifteen teams developing software for individual products or for separate areas of functionality for the same application. These areas or products may all need to release together, so an integrated end game is necessary. This does not mean that you leave the integration until the very end. Coordination with the other teams will be critical all along your development cycle, and if you have a test integration system, we recommend that you be sure that you have tried to integrate long before the end game.

您也可能有团队以外的考虑,例如,在企业级别使用外部团队交付的软件。

You also may have considerations beyond your team, for example, working with software delivered by external teams at the enterprise level.

利用这个游戏结束时间进行一些最终的探索性测试。退一步看整个系统,做一些端到端的场景。此类测试将确认应用程序是否正常工作,让您对产品更有信心,并为下一次迭代或发布提供信息。

Use this end-game time to do some final exploratory testing. Step back and look at the whole system and do some end-to-end scenarios. Such testing will confirm that the application is working correctly, give you added confidence in the product, and provide information for the next iteration or release.

测试发布候选

Testing the Release Candidate

我们建议对每个候选发布版本进行自动回归测试。如果您按照我们的建议对每个新构建持续运行自动回归测试,或者至少每天运行一次,那么您已经这样做了。如果您的某些回归测试是手动的,则您需要为这些测试计划时间,否则它们可能无法完成。如果有多个发布候选版本,则基于对每个构建所做的更改的风险评估将确定需要运行哪些测试。

We recommend that the automated regression testing be done against every release candidate. If you’re following our recommendation to run automated regression tests continually on each new build, or at least daily, you’ve already done this. If some of your regression tests are manual, you’ll need to plan time for those or they might not get done. A risk assessment based on changes made to each build will determine what tests need to be run if there is more than one release candidate.

在暂存环境中测试

Test on a Staging Environment

无论您使用的是传统开发流程还是敏捷开发流程,模拟生产的暂存环境对于发布前的最终测试以及测试发布流程本身都是至关重要的。作为最终游戏的一部分,您的应用程序应该部署到暂存区,就像您将其部署到生产环境中一样,或者就像您的客户在他们的环境中一样。在 Janet 见过的许多组织中,暂存环境通常在多个项目之间共享,部署必须作为发布计划的一部分进行安排。提前考虑如何处理依赖性、使用暂存环境与其他团队集成以及与外部第三方合作。这可能感觉像是“传统的”测试计划,但您可能正在与尚未采用敏捷开发的团队打交道。

Whether you are using traditional or agile development processes, a staging environment that mimics production is vital for final testing before release, as well as for testing the release process itself. As part of the end game, your application should be deployed to staging just like you would deploy it to production, or as your customers would on their environments. In many organizations that Janet has seen, the staging environment is usually shared among multiple projects, and the deployment must be scheduled as part of the release planning. Consider ahead of time how to handle dependencies, integrating with other teams using the staging environment, and working with external third parties. It might feel like “traditional” test planning, but you might be dealing with teams that haven’t embraced agile development.

尽管敏捷促进持续集成,但通常很难与项目控制之外的第三方产品或其他应用程序集成。暂存环境可以有更好的控制,以便外部应用程序可以连接并访问第三方测试环境。暂存环境还可用于负载和性能测试、模拟部署、故障转移测试以及手动回归测试和探索性功能测试。环境之间始终存在配置差异,因此您的暂存环境是测试这些的好地方。

Although agile promotes continuous integration, it is often difficult to integrate with third-party products or other applications outside your project’s control. Staging environments can have better controls so that external applications may connect and have access to third-party test environments. Staging environments can also be used for load and performance testing, mock deploys, fail-over testing, and manual regression tests and exploratory functional testing. There are always configuration differences between environments so your staging environment is a good place to test these.

最终非功能测试

Final Nonfunctional Testing

应在整个项目中针对您正在开发的应用程序的特定部分安排负载测试。如果您的暂存环境需求量很大,您可能无法在游戏结束前进行完整的系统负载测试。

Load testing should be scheduled throughout the project on specific pieces of the application that you are developing. If your staging environment is in high demand, you may not be able to do full system load testing until the end game.

到这个时候,您应该能够对所有产品功能进行长期运行的可靠性测试。检查正常负载下的崩溃和性能下降。在发布时完成时,它应该只是最终确认。

By this time, you should be able to do long-running reliability tests on all product functionality. Check for crashes and degradation of performance with normal load. When done at release time, it should be a final confirmation only.

容错和恢复测试也最好在您的暂存环境中完成,因为测试环境通常没有必要的设置。出于同样的原因,您可能只能测试某些方面的安全性。一个例子是 https,一种通过加密安全套接字的安全 http 连接。一些组织可能会选择仅在其暂存环境中拥有必要的证书。其他示例是集群或数据复制。确保让所有需要参与此测试的各方都参与进来。

Fault tolerance and recovery testing is best done on your staging environment as well, because test environments usually don’t have the necessary setup. For these same reasons, you may only be able to test certain aspects of security. One example is https, a secure http connection through encrypted secure sockets. Some organizations may choose to have the necessary certificates on their staging environment only. Other examples are clustering or data replication. Make sure you involve all parties who need to be included in this testing.

与外部应用程序集成

Integration with External Applications

您的团队可能敏捷,但您组织中的其他产品团队或与您的团队合作的第三方可能不是。

Your team may be agile, but other product teams in your organization, or third parties your team works with, may not be.

珍妮特的故事

Janet’s Story

在我合作过的一个组织中,批准信用卡的第三方合作伙伴有一个可以使用的测试帐户,但只能从临时环境访问。

In one organization that I worked with, the third-party partner that approved credit cards had a test account that could be used, but it was only accessible from the staging environment.

为了在开发过程中进行测试,创建了测试存根以根据所使用的信用卡号返回指定的结果。然而,这还不够,因为第三方有时会在我们不知道的情况下更改其功能。与实际的第三方进行测试对于项目的成功至关重要,也是最终游戏的关键部分。

To test during development, test stubs were created to return specified results depending on the credit card number used. However, this wasn’t sufficient because the third party sometimes changed functionality on its end that we weren’t aware of. Testing with the actual third party was critical to the success of the project, and it is a key part of the end game.

——珍妮特

—Janet

提前与其他产品团队或拥有需要与您的产品集成的产品的外部合作伙伴进行良好协调。如果您及早发现了这些风险并尽可能多地进行了前期测试,那么在游戏结束时进行的测试应该只是最终验证。但是,总有最后一刻的惊喜,因此您可能需要准备好对您的应用程序进行更改。

Coordinate well in advance with other product teams or outside partners that have products that need to integrate with your product. If you have identified these risks early and done as much up-front testing as possible, the testing done during the end game should be final verification only. However, there are always last-minute surprises, so you may need to be prepared to make changes to your application.

在开发过程中用于测试的模拟器和模拟对象等工具可以帮助减轻一些风险,但越早使用外部应用程序进行测试,风险就越低。

Tools like simulators and mock objects used for testing during development can help alleviate some of the risks, but the sooner you can test with external applications, the lower the risk.

数据转换和数据库更新

Data Conversion and Database Updates

在开发应用程序时,我们会更改字段、在数据库中添加列或删除过时的列。不同的团队以不同的方式解决这个问题。一些团队在每次新构建时都重新创建数据库。这适用于新应用程序,因为没有现有数据。但是,当应用程序存在于生产中并具有关联数据后,这种方法将不起作用。

As we are developing an application, we change fields, add columns in the database, or remove obsolete ones. Different teams tackle this in different ways. Some teams re-create the database with each new build. This works for new applications, because there is no existing data. However, after an application exists in production and has associated data, this approach won’t work.

应用程序需要考虑作为产品一部分的数据。与敏捷开发中的许多事情一样,需要数据库专家、程序员和测试人员在团队中共同努力,以确保成功发布数据库更改。Janet 看到了处理数据转换和向后兼容性的几种不同策略。当团队进行更改时,开发人员或数据库管理员可以创建数据库脚本。这些脚本成为构建的一部分,并不断进行测试。另一种选择是团队在完成所有数据库更改后对数据库运行“差异”。

An application needs to consider the data that is part of the product. As with so much in agile development, a joint effort by database experts, programmers, and testers on the team is required to ensure successful release of database changes. Janet has seen a couple of different tactics for dealing with data conversion and backward compatibility. Database scripts can be created by the developers or database administrators as the team makes changes. These scripts become part of the build and are continually tested. Another option is for the team to run “diffs” on the database after all of the database changes have been made.

如果您是测试人员,请让您的数据库管理员/开发人员帮助您的团队确保模式在生产、测试​​和暂存环境中保持一致。找到一种方法来保证在测试环境中所做的所有更改都将在发布期间在暂存和生产环境中完成。保持架构在列名、触发器、约束、索引和其他组件方面匹配(除了仍在开发中的新更改)。适用于编码和测试的同一学科也应适用于数据库开发和维护。

If you’re a tester, ask your database administrator/developer to help your team ensure that schemas are kept consistent among the production, testing, and staging environments. Find a way to guarantee that all changes made in the test environments will be done in the staging and production environments during release. Keep the schemas matching (except for the new changes still under development) in terms of column names, triggers, constraints, indices, and other components. The same discipline applied to coding and testing also should be applied to database development and maintenance.

丽莎的故事

Lisa’s Story

我们最近在生产中发布了一个错误,因为一些测试模式(包括回归测试使用的模式)缺少约束。没有适当的约束,代码不会失败。这引发了一项努力,以确保针对每个模式运行完全相同的更新脚本,以对给定版本进行更改。

We recently had a bug released to production because some of the test schemas, including the one used by regression tests, were missing a constraint. Without the constraint in place, the code didn’t fail. This triggered an effort to make sure the exact same update scripts get run against each schema to make changes for a given release.

事实证明,不同的测试模式存在细微差异,例如旧列仍保留在某些列中或不同模式中的列顺序不同,因此不可能在每个环境中运行相同的脚本。我们的数据库管理员带头努力重新创建所有测试模式以与生产完美兼容。他在每次迭代中创建一个脚本,其中包含所有必要的数据库更改,并在我们发布时在暂存和生产环境中运行相同的脚本。这看起来很简单,但是当您专注于交付新功能时,很容易错过细微的差别。

It turned out that different test schemas had small differences, such as old columns still remaining in some or columns in different order in different schemas, so it wasn’t possible to run the same script in every environment. Our database administrator led a major effort to re-create all of the test schemas to be perfectly compatible with production. He creates one script in each iteration with all necessary database changes and runs that same script in the staging and production environment when we release. This seems simple, but it’s easy to miss subtle differences when you’re focused on delivering new features.

——丽莎

—Lisa

自动化数据迁移可增强您测试它们的能力并减少人为错误的可能性。SQL、存储过程等原生数据库工具、SQL*Loader 和 bcp 等数据导入工具、shell 脚本和 Windows 命令文件可用于自动化,因为它们可以轻松克隆和更改。

Automating data migrations enhances your ability to test them and reduces the chance for human error. Native database tools such as SQL, stored procedures, data import tools such as SQL*Loader and bcp, shell scripts, and Windows command files can be used for automation because they can be cloned and altered easily.

无论数据库更新和转换脚本是如何创建或维护的,都需要对其进行测试。确保在更新脚本中捕获所有更改的最佳方法之一是使用可用的客户数据。客户习惯于以奇怪而美妙的方式使用应用程序,数据并不总是像我们希望的那样干净。如果开发团队清理数据库并对列施加额外限制,一旦查询触及与新限制不匹配的数据,客户站点上的应用程序可能会崩溃。您需要确保所做的任何更改仍与现有数据兼容。

No matter how the database update and conversion scripts are created or maintained, they need to be tested. One of the best ways to ensure all of the changes have been captured in the update scripts is to use the customer’s data if it is available. Customers have a habit of using the application in weird and wonderful ways, and the data is not always as clean as we would like it. If the development team cleans up the database and puts extra restrictions on a column, the application on the customer’s site might blow up as soon as a query touches a piece of data that does not match the new restrictions. You need to make sure that any changes you’ve made are still compatible with existing data.

丽莎的故事

Lisa’s Story

我的团队使用暂存环境来测试数据库更新脚本。脚本运行后,我们进行手动测试以验证所有更改和数据转换是否正确完成。我们的一些 GUI 测试脚本涵盖了一部分回归场景。这使我们对发布到生产环境充满信心,因为我们的测试能力更为有限。

My team uses the staging environment to test the database update scripts. After the scripts are run, we do manual testing to verify that all changes and data conversions completed correctly. Some of our GUI test scripts cover a subset of regression scenarios. This gives us confidence about releasing to production, where our ability to test is more limited.

——丽莎

—Lisa

在规划数据转换时,将数据清理视为缓解策略的一部分。您有机会获取以我们之前提到的一些“奇怪而美妙”的方式输入的数据,并对其进行修改或操作,使其符合新的约束条件。这种类型的工作可能需要很长时间才能完成,但在维护数据完整性方面通常非常值得。

When planning a data conversion, think about data cleanup as part of the mitigation strategy. You have the opportunity to take the data that was entered in some of the “weird and wonderful” ways we mentioned before and massage or manipulate it so it conforms to the new constraints. This type of job can take a long time to do but is often very worthwhile in terms of maintaining data integrity.

不是每个人都能在登台环境中对生产数据进行足够好的模拟。如果客户的数据不可用,缓解策略是在客户站点安装 UAT。另一种降低风险的方法是尽量避免大规模更新并在较小的阶段发布。在开发旧功能的同时开发新功能,并使用系统属性来“开启”其中一个功能。旧功能可以继续在生产中工作,直到新功能完成。同时,可以在每次迭代时对新代码进行测试。可以在不影响旧代码的情况下将新的列和表添加到生产表中,从而最大限度地减少最终发布的数据迁移或转换。

Not everyone can do a good enough simulation of production data in the staging environment. If a customer’s data is not available, a mitigation strategy is to have a UAT at the customer site. Another way to mitigate risk is to try to avoid large-scale updates and release in smaller stages. Develop new functionality in parallel with the old functionality and use a system property to “turn on” one or the other. The old functionality can continue to work in production until the new functionality is complete. Meanwhile, testing can be done on the new code at each iteration. New columns and tables can be added to production tables without affecting the old code so that the data migration or conversion for the final release is minimized.

安装测试

Installation Testing

组织通常有一个单独的团队部署到生产或创建产品集。这些团队成员应该有机会完全按照生产环境进行部署。如果他们使用部署到暂存作为他们的试验场,他们可以在向客户发布之前解决任何问题。

Organizations often have a separate team that deploys to production or creates the product set. These team members should have the opportunity to practice the deployment exactly as they would for production. If they use the deployment to staging as their proving ground, they can work out any of the problems long before they release to the customer.

测试产品安装还可以意味着测试不同操作系统或硬件的收缩包装产品的各种安装。产品表现如何?它能达到预期的效果吗?系统需要关闭多长时间才能安装?我们可以在不中断的情况下进行部署吗?我们能否让用户体验尽可能愉快?

Testing product installations can also mean testing various installations of shrink-wrapped products to different operating systems or hardware. How does the product behave? Does it do what is expected? How long will the system need to be down for installation? Can we deploy without taking an outage? Can we make the user experience as pleasant as possible?

珍妮特的故事

Janet’s Story

不久前我有一次不太愉快的经历,这让我希望有人在我发现问题之前已经测试并修复了它。我买了一台新笔记本电脑,想将我的一个应用程序的许可证转移到新计算机上。它带有同一应用程序的试用版,因此转移应该很容易,但新 PC 无法识别产品密钥——它一直说它无效。我打电话给支持台,经过一些诊断后,我被告知它们被认为是不同的产品,因此密钥无法使用。

I had an experience a while ago that was not so pleasant, and it led me to wish that someone had tested and fixed the issue before I found it. I bought a new laptop and wanted to transfer my license for one of my applications to the new computer. It came with a trial version of the same application, so the transfer should have been easy, but the new PC did not recognize the product key—it kept saying it was invalid. I called the support desk and after a bit of diagnostics, I was informed they were considered different products, so the key wouldn’t work.

又多了两个小时的支持时间,问题就解决了。必须删除试用版,必须重新安装旧版本,必须重新输入密钥,并且必须上传自最初购买以来的所有更新。开发团队测试该场景并向客户提供信息性消息“试用版与您的产品密钥不兼容”会容易得多。这样的消息会让我自己找出问题并解决它,而不是占用支持人员的时间。

Two more hours of support time, and the issue was fixed. The trial version had to be removed, an old version had to be reinstalled, the key had to be reentered, and all updates since the original purchase had to be uploaded. How much easier would it have been for the development team to test that scenario and offer the customer an informative message saying, “The trial version is not compatible with your product key.” A message such as that would have let me figure out the problem and solve it myself rather than taking the support person’s time.

——珍妮特

—Janet

花点时间确定测试安装的要求。如果你让你的客户满意,那最终是值得的。

Take the time you need to determine what your requirements are for testing installation. It will be worth it in the end if you satisfy your customers.

沟通

Communication

不同开发团队成员之间的持续沟通始终很重要,但在我们完成发布时尤为重要。如果需要,召开额外的站立会议,以确保一切都为发布做好准备。如果有可能忘记某些步骤,请为发布任务写卡片。

Constant communication between different development team members is always important, but it’s especially critical as we wrap up the release. Have extra stand-up meetings, if needed, to make sure everything is ready for the release. Write cards for release tasks if there’s any chance some step might be forgotten.

丽莎的故事

Lisa’s Story

我的团队在每次迭代后发布。我们通常会在冲刺的最后一个下午快速站会,以触及底线并找出任何未解决的问题。在团队进行大量发布实践之前,我们编写了发布任务卡,例如“在暂存中运行数据库更新脚本”和“在生产中验证数据库更新”。有了更多的部署经验,我们不再需要这些卡片,除非我们有一个新的团队成员可能需要额外的提醒。不过,拥有用于发布任务的卡片永远不会有坏处。

My team releases after each iteration. We usually have a quick stand-up on the last afternoon of the sprint to touch base and identify any loose ends. Before the team had a lot of practice with releases, we wrote release task cards such as “run database update script in staging” and “verify database updates in production.” With more experience at deploying, we no longer need those cards unless we have a new team member who might need an extra reminder. It never hurts to have cards for release tasks, though.

——丽莎

—Lisa

任务提醒,无论是在完整的实施计划中,还是像 Lisa 的团队那样只是写在任务卡上,通常都是必要的。在简单的实现中,白板效果很好。

Reminders of tasks, whether they are in a full implementation plan or just written on task cards as Lisa’s team does, are often necessary. On simple implementations, a whiteboard works well.

如果还没有准备好怎么办?

What If It’s Not Ready?

通过以多种形式不断跟踪进度,例如构建、回归测试套件、故事板和燃尽图,团队通常可以在发布遇到麻烦时提前知道。有时间放下故事并重新调整。尽管如此,最后一刻的灾难还是可能发生。如果构建机器在迭代的最后一天坏了怎么办?如果测试数据库崩溃导致最终测试无法完成怎么办?如果直到最后的功能测试才检测到 showstopper 错误怎么办?

By constantly tracking progress in many forms, such as builds, regression test suites, story boards, and burndown charts, a team usually knows well in advance when it’s in trouble on a release. There’s time to drop stories and readjust. Still, last-minute disasters can happen. What if the build machine breaks on the last day of the iteration? What if the test database crashes so that final testing can’t be completed? What if a showstopper bug isn’t detected until final functional testing?

我们强烈建议不要在迭代中增加额外的天数,因为它会占用下一个迭代或发布开发。一个经验丰富的团队可能足够灵活,可以做到这一点,但它可能会破坏一个新团队。然而,危急时刻需要采取危急措施。如果您每两周发布一次,您可以简单地跳过实际发布,将时间安排到下一次迭代中以更正问题并完成,然后在下一个预定日期发布。如果测试任务被推迟或忽略而发布继续进行,请向团队提出这个问题。测试需求是否发生变化,或者团队是否在冒险并牺牲质量以满足最后期限?如果交付日期已确定且处于危险之中,团队应缩减发布范围。

We strongly advise against adding extra days to an iteration, because it will eat into the next iteration or release development. An experienced team might be flexible enough to do this, but it can derail a new team. Still, desperate times call for desperate measures. If you release every two weeks, you may simply be able to skip doing the actual release, budget time into the next iteration to correct the problems and finish up, and release on the next scheduled date. If testing tasks are being put off or ignored and the release goes ahead, bring up this issue with the team. Did the testing needs change, or is the team taking a chance and sacrificing quality to meet a deadline? The team should cut the release scope if the delivery date is fixed and in jeopardy.

如果您的发布周期更长,更像是三个月,您应该提前知道您的发布是否处于危险之中。您可能已经计划了至少两周的结束游戏,这将仅用于最终验证。当您有一个较长的发布周期时,您就有更多的时间来确定您应该做什么,无论是删除功能还是更改时间表。

If your release cycle is longer, more like three months, you should know in advance if your release is in jeopardy. You probably have planned an end game of at least two weeks, which will just be for final validation. When you have a longer release cycle, you have more time to determine what you should do, whether it’s dropping functionality or changing the schedule.

如果您的组织需要在固定的日期发布某些功能,并且最后一刻的故障威胁到发布,请评估您的备选方案。看看您是否可以继续相同的开发周期,但将发布本身推迟一天或一周。也许有问题的代码可以暂时退出,稍后再打补丁。客户对什么对企业有用有最终决定权。

If your organization requires certain functionality to be released on a fixed day and last-minute glitches threaten the release, evaluate your alternatives. See if you can continue on your same development cycle but delay the release itself for a day or a week. Maybe the offending piece of code can be backed out temporarily and a patch done later. The customers have the ultimate say in what will work for the business.

丽莎的故事

Lisa’s Story

在极少数情况下,当我们的团队遇到最后关头的问题时,我们会根据情况使用不同的方法。如果现在没有必须发布的关键内容,我们有时会跳过发布并在下一个发布日发布两个迭代的价值。如果必须加入某些关键内容,我们会延迟一两天发布。有时我们可以继续发布我们拥有的东西,并在第二天发布补丁。有一次,我们决定有一个特殊的一周迭代来纠正问题,发布,然后回到正常的两周迭代计划。

On the rare occasions when our team has faced the problem of last-minute showstoppers, we’ve used different approaches according to the situation. If there’s nothing critical that has to be released right now, we sometimes skip the release and release two iterations’ worth on the next release day. If something critical has to go in, we delay the release a day or two. Sometimes we can go ahead and release what we have and do a patch release the next day. On one occasion, we decided to have a special one-week iteration to correct the problems, release, and then go back to the normal two-week iteration schedule.

经过四年多的敏捷开发实践,我们几乎 100% 的时间都有稳定的构建,我们对能够在必要时发布有信心。我们需要大量的纪律和持续改进我们的流程,以便感觉到更灵活的方法可以为我们工作。如果可以的话,能够尽早发布一些有价值的功能也很好。我们努力避免的是陷入死亡漩涡,我们永远无法按计划发布,我们总是在追赶。

After more than four years of practicing agile development, we have a stable build almost 100% of the time, and we feel confident about being able to release whenever it’s necessary. We needed a lot of discipline and continual improvement to our process in order to feel that a more flexible approach could work for us. It’s also nice to be able to release a valuable bit of functionality early, if we can. What we’ve worked hard to avoid is falling into a death spiral where we can never release on schedule and we’re always playing catch-up.

如果不能按时发布,请不要自责。你的团队正在尽力而为。花时间分析为什么你落后于计划或过度承诺,​​并采取行动防止它再次发生。

Don’t beat yourself up if you can’t release on time. Your team is doing its best. Do spend time analyzing why you got behind schedule, or over-committed, and take action to keep it from happening again.

——丽莎

—Lisa

通过良好的规划、密切协作、通过测试驱动编码以及在编码时进行测试,努力防止出现“不行”的情况。如果您的跟踪显示该版本可能处于危险之中,请尽可能删除无法完成的功能。如果发生不好的和意想不到的事情,不要惊慌。让整个团队和客户团队参与进来,集思广益,找出最佳解决方案。

Work to prevent a “no go” situation with good planning, close collaboration, driving coding with tests, and testing as you code. If your tracking shows the release could be in jeopardy, remove the functionality that can’t be finished, if possible. If something bad and unexpected happens, don’t panic. Involve the whole team and the customer team, and brainstorm about the best solution.

客户测试

Customer Testing

有几种不同的方式可以让您的客户参与进来以获得他们的认可或反馈。用户验收测试可以相当正式,需要企业签字。它表示接受发布。Alpha 或 Beta 测试是一种获取有关您要发布但尚未准备就绪的产品的反馈的方法。

There are a couple of different ways in which to involve your customers to get their approval or feedback. User Acceptance Testing can be fairly formal, with sign-offs from the business. It signifies acceptance of a release. Alpha or beta testing is a way to get feedback on a product you are looking to release but which is not quite ready.

UAT

UAT

用户验收测试 (UAT) 在大型定制应用程序和内部应用程序中都很重要。它由所有受影响的业务部门执行,以验证系统的可用性并确认系统的现有和新的(强调新的)业务功能。你的客户是那些必须使用该应用程序的人,因此他们需要确保它可以在他们的系统和数据上正常工作。

User Acceptance Testing (UAT) is important in large customized applications as well as internal applications. It’s performed by all affected business departments to verify usability of the system and to confirm existing and new (emphasis on new) business functionality of the system. Your customers are the ones who have to live with the application, so they need to make sure it works on their system and with their data.

在前面的章节中,我们经常谈到让客户尽早参与,但在那个时候,测试是针对正在开发的特定功能进行的。UAT 通常在团队决定质量足以发布后完成。但有时,时间表决定了发布周期。如果是这种情况,请尝试将 UAT 周期向上移动以与您的最终游戏并行运行。应用程序应该足够稳定,以便您的团队可以在部署到暂存系统的同时部署到客户的测试系统。

In previous chapters we’ve often talked about getting the customers involved early, but at those times, the testing is done on specific features under development. UAT is usually done after the team decides the quality is good enough to release. Sometimes though, the timeline dictates the release cycle. If that is the case, then try moving the UAT cycle up to run parallel with your end game. The application should be stable enough so that your team could deploy to the customer’s test system at the same time as they deploy to staging.

珍妮特的故事

Janet’s Story

在我加入的一个团队中,客户非常挑剔。事实上,这是我见过的最挑剔的。他们总是要求进行整整一周的 UAT,以确保他们有时间进行所有测试。他们准备了测试用例并检查了所有内容,包括英语和法语的所有内容。Showstopper 错误包括拼写错误,例如法语内容中缺少重音。随着时间的推移,随着他们对我们的发布越来越有信心并且发现的错误越来越少,他们放宽了他们的要求但仍然需要一周时间,以防他们无法立即完成。他们的事业群很忙。

In one team I joined, the customers were very picky. In fact, the pickiest I had ever seen. They always asked for a full week of UAT just to be sure they had the time to test it all. They had prepared test cases and checked them all, including all the content, both in English and in French. Showstopper bugs included spelling errors such as a missing accent in the French content. Over time, as they gained more confidence in our releases and found fewer and fewer errors, they relaxed their demands but still wanted a week, just in case they couldn’t get to it right away. Their business group was very busy.

一个版本的发布推迟了时间线。我们被推迟到发布日期,但无法获得所有功能,并为最终游戏留出两周时间。我们与商业用户进行了交谈,我们决定将结束游戏时间缩短到一周;当项目团队完成系统测试和清理时,业务用户将执行他们的 UAT。我们能够做到这一点的唯一原因是客户对我们团队的信任以及我们发布的一致性。

One release came that pushed the timeline. We were being held to the release date but couldn’t get all the functionality in and leave two weeks for the end game. We talked with the business users and we decided to decrease the end game to one week; the business users would perform their UAT while the project team finished up their system testing and cleanup. The only reason we were able to do this was because of the trust the customer had in our team and the consistency of our releases.

好消息是,UAT 再一次没有发现不能等到下一个版本的问题。

The good news was that, once again, the UAT found no issues that could not wait until the next release.

——珍妮特

—Janet

图 20-1显示了在发布周期结束时具有正常 UAT 的示例时间线。团队开始着手下一个版本,进行发布计划,并在所有团队成员准备就绪的情况下开始第一次迭代。

Figure 20-1 shows an example timeline with a normal UAT at the end of the release cycle. The team starts working on the next release, doing release planning, and starts the first iteration with all team members ready to go.

图 20-1使用 UAT 的发布时间表

Figure 20-1 Release timeline with UAT

图像

与客户合作,让他们了解流程、他们的角色以及对他们的期望。如果 UAT 不顺利,则可能需要高水平的支持。一个有经验的客户测试团队可能已经定义了测试用例,但大多数时候它的测试是临时的。客户可能会像对待日常工作一样对待他们的测试,但可能会专注于新功能。这是一个观察人们如何使用该系统并从他们那里获得有关哪些方面运作良好以及哪些改进对他们有帮助的反馈。

Work with customers so that they understand the process, their role, and what is expected of them. If the UAT is not smooth, then the chances are there will be a high level of support needed. An experienced customer test team may have defined test cases, but most often its testing is ad hoc. Customers may approach their testing as if they were doing their daily job but will probably focus on the new functionality. This is an opportunity to observe how people use the system and to get feedback from them on what works well and what improvements would help them.

测试人员可以通过审查测试运行和记录的缺陷,并通过跟踪缺陷直至完成,为执行 UAT 的客户提供支持。我们都发现向参与 UAT 的客户提供开发期间完成的所有测试的报告以及结果很有帮助。这有助于他们决定将自己的测试集中在哪里。

Testers can provide support to the customers who are doing the UAT by reviewing tests run and defects logged, and by tracking defects to completion. Both of us have found it helpful to provide customers involved in doing UAT with a report of all of the testing done during development, along with the results. That helps them decide where to focus their own testing.

阿尔法/贝塔测试

Alpha/Beta Testing

如果您是一个向大型客户群分发软件的组织,您可能没有正式的 UAT。您更有可能加入 alpha 或 beta 测试。您的团队希望从您的真实客户那里获得对新功能的反馈,这是一种这样做的机制。Alpha 测试是软件新版本的早期分发。由于可能存在一些重大错误,因此您需要明智地挑选客户。如果您选择这种客户反馈方法,请确保您的客户了解他们的角色。Alpha 测试是为了获得对功能的反馈——而不是报告错误。

If you are an organization that distributes software to a large customer base, you may not have a formal UAT. You are much more likely to incorporate alpha or beta testing. Your team will want to get feedback on new features from your real customers, and this is one mechanism for doing so. Alpha testing is early distribution of new versions of software. Because there are likely to be some major bugs, you need to pick your customers wisely. If you choose this method of customer feedback, make sure your customers understand their role. Alpha testing is to get feedback on the features—not to report bugs.

Beta 测试更接近 UAT。预计该版本相当稳定,可以实际使用。对于大多数客户来说,它可能还没有“准备好迎接黄金时段”,但许多客户可能觉得新功能值得冒险。顾客应该了解这不是正式版本,您要求他们测试您的产品并报告错误。

Beta testing is closer to UAT. It is expected that the release is fairly stable and can actually be used. It may not be “ready for prime time” for most customers, but many customers may feel the new features are worth the risk. Customers should understand that it is not a formal release and that you are asking them to test your product and report bugs.

作为测试人员,了解客户如何看待产品很重要,因为它可能会影响您的测试方式。Alpha 和 Beta 测试可能是您与最终用户互动的唯一机会,因此请利用这个机会了解产品如何满足他们的需求。

As a tester, it is important to understand how customers view the product, because it may affect how you test. Alpha and beta testing may be the only time you get to interact with end users, so take advantage of the chance to learn how well the product meets their needs.

开发后测试周期

Post-Development Testing Cycles

如果您在大型组织工作或正在开发大型复杂系统的组件,您可能需要在开发完成后安排时间进行测试。有时 UAT 测试或测试协调并不像预期的那样顺利,因此时间线会延长。包含所有生产系统的测试版本的测试环境可能仅在较小的预定时间窗口内可用。您可能需要与处理与您交互的其他应用程序的团队协调测试会话。不管是什么原因,您都需要额外的测试时间,其中不包括整个开发团队。

If you work in a large organization or are developing a component of a large, complex system, you may need to budget time for testing after development is complete. Sometimes the UAT testing, or the test coordination, isn’t as smooth as it could be, so the timeline stretches out. Test environments that include test versions of all production systems may only be available for small, scheduled windows of time. You may need to coordinate test sessions with teams working on other applications that interact with yours. Whatever the reason, you need extra testing time that does not include the whole development team.

丽莎的故事

Lisa’s Story

我在一个团队工作,为一家大型电信客户开发内部和外部应用程序的组件。我们只能按预定的时间间隔访问完整的测试环境。发布时间也很紧凑。

I worked on a team developing components of both internal and external applications for a large telecom client. We could only get access to the complete test environment at scheduled intervals. Releases were also tightly scheduled.

开发团队每两周进行一次迭代。它只能在每三次迭代后发布到测试环境。当时有两周的系统集成和用户验收测试周期,紧接着就是发布。

The development team worked in two-week iterations. It could release to the test environment only after every third iteration. At that time, there was a two-week system integration and user acceptance test cycle, followed by the release.

我团队的某个人需要指导后期开发测试阶段。与此同时,开发人员开始了新功能的新迭代,他们需要一名测试人员来帮助完成这项工作。

Someone from my team needed to direct the post-development testing phase. Meanwhile, the developers were starting a new iteration with new features, and they needed a tester to help with that effort.

团队必须付出特别的努力来确保担任测试员角色的人从头到尾跟进每个版本。例如,我从头到尾完成第 1 版的工作。当团队开始第 2 版的第一次迭代时,Shauna 接管了测试员角色,而我负责协调第 1 版的系统测试和 UAT。Shauna 一直担任主要测试员第 2 版,而我担任第 3 版的角色。

The team had to make a special effort to make sure someone in the tester role followed each release from start to finish. For example, I worked from start to finish on release 1. Shauna took over the tester role as the team started work on the first iteration of release 2, while I was coordinating system testing and UAT on release 1. Shauna stayed as primary tester for release 2, while I assumed that role for release 3.

——丽莎

—Lisa

图 20-2显示了扩展 UAT 的时间线示例。发生这种情况的原因有很多,而且问题可能并不总是出在 UAT 上。大多数团队都准备好开始下一个版本的工作,但通常测试人员仍在与客户合作,完成最终测试。有时,程序员也会参与其中。有几个选项。如果团队足够大,您可以开始下一个版本,而几个团队成员使用现有版本(版本 2—图 20-2中的备选方案 2 )。如果您有一个小团队,您可能需要考虑迭代 0,让程序员在新功能上进行重构或尖峰(实验),这样与客户一起工作的测试人员就不会落后(版本 2—图 20 中的备选方案1- 2 ).

Figure 20-2 shows an example timeline where the UAT was extended. This could happen for any number of reasons, and the issue may not always be UAT. Most of the team is ready to start working on the next release, but often a tester is still working with customers, completing final testing. Sometimes a programmer will be involved as well. There are a couple of options. If the team is large enough, you can probably start the next release while a couple of team members work with the existing release (Release 2—Alternative 2 in Figure 20-2). If you have a small team, you may need to consider an Iteration 0 with programmers doing refactoring or spikes (experiments) on new functionality so that the tester working with the customer does not get left behind (Release 2—Alternative 1 in Figure 20-2).

图 20-2发布时间线——使用扩展 UAT 的替代方法

Figure 20-2 Release timeline—alternative approach with extended UAT

图像

创造性地处理项目现实强加给你的团队的情况。虽然计划很少会按预期进行,但提前计划仍然可以帮助您确保合适的人员到位以及时交付产品。

Be creative in dealing with circumstances imposed on your team by the realities of your project. While plans rarely work as expected, planning ahead can still help you make sure the right people are in place to deliver the product in a timely manner.

可交付成果

Deliverables

在本章的第一节中,我们讨论了产品的构成要素。这个问题的答案实际上取决于听众:谁在接受产品,他们的期望是什么?

In the first section of this chapter we talked about what makes a product. The answer to this will actually depend on the audience: Who is accepting the product, and what are their expectations?

如果您的客户需要满足 SOX (Sarbanes-Oxley) 合规性要求,则需要某些可交付成果。例如,与 Janet 合作的一位客户认为测试结果应该完整记录,并将测试结果作为他们的 SOX 合规性测量点之一,而另一位客户根本没有测量测试结果。与合规和审计人员合作,在您开始项目时确定报告需求。

If your customers need to meet SOX (Sarbanes-Oxley)compliance requirements, there will be certain deliverables that are required. For example, one customer Janet has worked with felt test results should be thoroughly documented, and made test results one of their SOX compliance measurement points, while a different customer didn’t measure test results at all. Work with compliance and audit personnel to identify reporting needs as you begin a project.

多少文档就足够了?珍妮特在回答这个问题之前总是问两个问题:“它是给谁的?” 和“他们用它做什么?” 如果这些问题没有充分的答案,那么请考虑是否真的需要该文档。

How much documentation is enough? Janet always asks two questions before answering that question: “Who is it for?” and “What are they using it for?” If there are no adequate answers to those questions, then consider whether the documentation is really needed.

可交付成果并不总是针对最终客户的,也不总是以软件的形式出现。有很多内部客户,例如生产支持团队成员。他们需要什么来使他们的工作更轻松?工作流程图可以帮助他们理解新功能。他们可能想知道是否有适当的变通办法,以便他们可以帮助客户解决问题。

Deliverables are not always for the end customer, and they aren’t always in the form of software. There are many internal customers, such as the production support team members. What will they need to make their job easier? Workflow diagrams can help them understand new features. They would probably like to know if there are work-arounds in place so they can help customers through problems.

Janet 经常被问及代码的测试覆盖率,通常是管理层问的。单元测试或回归测试正在测试多少应用程序?问题是这个数字本身只是一个数字,它可能高或低的原因有很多。此外,代码覆盖率不会告诉您可能遗漏的功能,因为这些功能尚不存在代码。可交付成果(例如代码覆盖率)的受众不应该是管理层,而应该是团队本身。它可以用来查看代码的哪些区域没有被测试。

Janet often gets asked about test coverage of code, usually by management. How much of the application is being tested by the unit tests or regression tests? The problem is that the number by itself is just a number, and there are so many reasons why it might be high or low. Also, code coverage doesn’t tell you about features that might have been missed, for which no code exists yet. The audience for a deliverable such as code coverage should not be management, but the team itself. It can be used to see what areas of the code are not being tested.

培训也可以被视为一种可交付成果。许多应用程序需要为客户定制培训课程。其他人可能只需要在线帮助或用户手册。培训可以决定您的产品是否成功,因此考虑这一点很重要。Lisa 的团队经常为测试人员或产品负责人编写任务卡,以确保安排好培训材料和课程。有些人可能觉得培训不是测试人员或开发团队中任何其他人的工作。然而,敏捷团队的目标是与业务尽可能紧密地合作。测试人员通常拥有领域专业知识,至少能够确定新功能或更新功能可能需要的培训。即使培训不是测试人员的责任,如果企业没有计划培训课程,她也可以提出问题。

Training could be considered a deliverable as well. Many applications require customized training sessions for customers. Others may only need online help or a user manual. Training could determine the success of your product, so it’s important to consider. Lisa’s team often writes task cards for either a tester or the product owner to make sure training materials and sessions are arranged. Some people may feel training isn’t the job of testers or anyone else on the development team. However, agile teams aim to work as closely as possible with the business. Testers often have the domain expertise to be able to at least identify training that might be needed for new or updated features. Even if training isn’t the tester’s responsibility, she can raise the issue if the business isn’t planning training sessions.

许多敏捷团队都有技术作家作为团队的一部分,负责编写在线帮助或电子形式的文档。甚至包括一个应用程序帮助入门的培训视频,团队的不同成员是培训师。创造一个成功的产品是团队的责任。

Many agile teams have technical writers as part of the team that write online help or electronic forms of documentation. One application even included training videos to help get started, and different members of the team were the trainers. It is the responsibility of the team to create a successful product.

发布产品

Releasing the Product

当我们谈论发布产品时,我们的意思是让客户可以使用任何可能采用的格式。您的组织可能有一个不断更新的网站或一个交付给少数大客户的自定义应用程序。也许该产品采用收缩包装并交付给全球数百万台 PC,或者从 Internet 下载。

When we talk about releasing the product, we mean making it available to the customer in whatever format that may take. Your organization might have a website that gets updated or a custom application that is delivered to a few large customers. Maybe the product is shrink-wrapped and delivered to millions of PCs around the world, or downloaded off the Internet.

发布验收标准

Release Acceptance Criteria

你怎么知道什么时候完成?验收标准是定义何时接受产品的传统方式。绩效标准可能有被满足。我们在每次迭代开始时为每个故事捕获这些,我们也可以在开始主题或史诗时为更大的功能集指定它们。客户可以设置质量标准,例如自动测试覆盖一定百分比的代码,或者某些测试必须通过。零严重错误或对系统有严重影响的零错误等行项目通常是发布标准的一部分。客户需要决定他们如何知道产品何时具有足够的价值。测试人员可以帮助他们定义实现目标的发布标准。

How do you know when you’re done? Acceptance criteria are a traditional way of defining when to accept the product. Performance criteria may have to be met. We capture these for each story at the start of each iteration, and we may also specify them for larger feature sets when we begin a theme or epic. Customers may set quality criteria such as a certain percentage of code covered by automated tests, or that certain tests must pass. Line items such as having zero critical bugs, or zero bugs with serious impact to the system, are often part of the release criteria. The customers need to decide how they’ll know when there’s enough value in the product. Testers can help them define release criteria that accomplish their goals.

敏捷团队努力实现质量目标的精神,而不仅仅是文字。他们不会将错误的严重性降级为中等,因此他们可以说他们达到了没有高严重性错误的标准。相反,他们经常查看错误趋势并想办法确保高严重性错误不会在生产中出现。

Agile teams work to attain the spirit of the quality goals, not just the letter. They don’t downgrade the severity of bugs to medium so they can say they achieved the criterion of no high-severity bugs. Instead, they frequently look at bug trends and think of ways to ensure that high-severity bugs don’t occur in production.

您应该预先与客户协商您的质量水平,以免出现不愉快的意外。您的团队和客户使用真实示例定义的验收测试应作为发布进度的里程碑。如果您的客户对错误的容忍度非常低,并且 100% 的验收测试必须通过,那么您的迭代速度应该考虑到这一点。如果新功能比错误修复更重要,那么,也许你会在发布时出现错误。

Your quality level should be negotiated with your customer up front so that there are no unpleasant surprises. The acceptance tests your team and your customers defined, using real examples, should serve as milestones for progress toward release. If your customer has a very low tolerance for bugs, and 100% of those acceptance tests must be passing, your iteration velocity should take that into consideration. If new features are more important than bug fixes, well, maybe you will be shipping with bugs.

每个项目、每个团队、每个业务都是独一无二的。敏捷团队与业务专家合作,决定何时准备好将软件交付生产。如果发布截止日期一成不变,企业将不得不修改范围。如果在软件具有足够的价值时有足够的灵活性来发布,则团队可以决定何时满足质量标准并且软件可以投入生产。

Each project, each team, each business is unique. Agile teams work with the business experts to decide when they’re ready to deliver software to production. If the release deadline is set in stone, the business will have to modify scope. If there’s enough flexibility to release when the software has enough value, the teams can decide when the quality criteria have been met and the software can go to production.

传统软件开发的工作时间很长,截止日期提前很多,从一个阶段到下一个阶段需要清除障碍。敏捷开发让我们能够以小增量生产高质量的软件,并在必要时发布。开发和客户团队可以密切合作来定义和决定发布什么以及何时发布。测试人员可以在此目标设定过程中发挥关键作用。

Traditional software development works in long time frames, with deadlines set far in advance and hurdles to clear from one phase to the next. Agile development lets us produce quality software in small increments and release as necessary. The development and customer teams can work closely to define and decide what to release and when. Testers can play a critical role in this goal-setting process.

发布管理

Release Management

许多组织都有发布管理团队,但如果您没有,仍然会有人做这项工作。在小型组织中,很多时候都是 QA 经理担任此角色。负责发布的人员可以与利益相关者召开发布准备会议以评估准备情况。

Many organizations have a release management team, but if you don’t, someone still does the work. Many times in a small organization it is the QA manager who fulfills this role. The person leading the release may hold a release readiness meeting with the stakeholders to evaluate readiness.

发布准备清单是一个很好的工具,可以用来检查对你的团队来说重要的事情。此清单的目的是帮助团队客观地确定已完成的内容并确定与未完成任务相关的风险。

A release readiness checklist is a great tool to use to walk through what is important to your team. The intention of this checklist is to help the team objectively determine what was completed and identify the risks associated with not completing a task.

例如,如果不需要培训因为对产品所做的更改对最终用户是透明的,那么风险就很低。但是,如果在系统中创建新用户的过程发生重大变化,生产支持或帮助团队面临的风险将非常高,并且可能需要延迟。必须考虑所有利益相关者的需求。

For example, if training is not required because the changes made to the product were transparent to the end user, then the risk is low. However, if there were significant changes to the process for how a new user is created in the system, the risk would be very high to the production support or help teams, and may warrant a delay. The needs of all stakeholders must be considered.

版本说明对于任何产品版本都很重要。这些的形式取决于听众。如果您的产品是针对开发人员的,那么“read me”文本文件可能就可以了。在其他情况下,您可能想让它们更正式。无论是何种媒体,它们都应该满足受众的需求。不要提供很多不需要的附加信息。

Release notes are important for any product release. The formality of these depends on the audience. If your product is aimed at developers, then a “read me” text file is probably fine. In other cases, you may want to make them more formal. Whatever the media, they should address the needs of the audience. Don’t provide a lot of added information that isn’t needed.

当 Janet 获得新版本时,她做的第一件事就是检查版本和所有组件。“我得到了他们所说的给我的东西吗?安装前是否需要考虑特殊说明,例如依赖项或升级脚本?” 这些是发行说明中需要回答的简单问题。其他要包括的内容是客户应该寻找的新功能。

When Janet gets a new release, one of the first things she does is check the version and all of the components. “Did I get what they said they gave me? Are there special instructions I need to consider before installing, such as dependencies or upgrade scripts?” Those are good simple questions to answer in release notes. Other things to include are the new features that the customer should look for.

发行说明应特别考虑不属于您的开发团队交付内容的组件,例如由不同团队准备的帮助文件或用户手册。有时旧的发行说明会留在发行媒体上,这对最终用户可能有用也可能没用。考虑什么适合您的团队和您的应用程序。

Release notes should give special consideration to components that aren’t part of what your development team delivered, such as a help file or user manuals prepared by a different team. Sometimes old release notes get left on the release media, which may or may not be useful to the end user. Consider what is right for your team and your application.

包装

Packaging

我们已经就持续集成谈了很多。我们倾向于认为这是理所当然的,而忘记了良好的配置管理意味着什么。“一次构建,多次部署”是让我们在发布时充满信心的部分原因。我们知道我们在暂存中测试的构建与客户测试的构建相同在 UAT 中,是我们发布到生产环境的构建。这对于成功发布至关重要。

We’ve talked a lot about continual integration. We tend to take it for granted and forget what good configuration management means. “Build once, deploy multiple times” is part of what gives us confidence when we release. We know that the build we tested in staging is the same build that the customer tested in UAT and is the build we release to production. This is critical for a successful release.

如果该产品是为外部客户准备的,那么安装应该很容易,因为安装可能是客户首先看到的产品。了解您的听众及其对错误的容忍度。产品将如何交付?例如,如果要从 Internet 下载它,那么它应该是一个简单的下载和安装。如果它是一个庞大的企业系统,那么您的组织可能需要派一名技术支持人员来帮助安装该产品。

If the product is intended for an external customer, the installation should be easy, because the installation may be the first look at the product that customer has. Know your audience and its tolerance level for errors. How will the product be delivered? For example, if it is to be downloaded off the Internet, then it should be a simple download and install. If it is a huge enterprise system, then maybe your organization needs to send a support person with the product to help with the install.

客户期望

Customer Expectations

在我们向客户推出新软件之前,我们最好确定他们已经准备好使用它。我们必须确保他们知道期望的新功能是什么,并且他们有一些方法来处理出现的问题。

Before we spring new software on our customers, we’d better be certain they are ready for it. We must be sure they know what new functionality to expect and that they have some means to deal with problems that arise.

生产支持

Production Support

许多组织都有一个生产或运营支持团队,负责维护代码并在代码投入生产后为客户提供支持。如果您的公司有生产支持团队,那么该团队就是您的第一个客户。让它成为你的伙伴。生产支持团队收到来自客户的缺陷报告和增强请求,他们可以与您的团队一起识别高风险区域。

Many organizations have a production or operations support team that maintains the code and supports customers after it’s in production. If your company has a production support team, that group is your first customer. Make it your partner as well. Production support teams receive defect reports and enhancement requests from the customers, and they can work with your team to identify high-risk areas.

很多时候,生产支持团队是接受开发团队发布的团队。如果您的组织有这种类型的交接,那么您的开发团队与生产支持团队密切合作以使其平稳过渡非常重要。确保生产支持团队了解如何使用系统的日志文件以及消息和监控系统,以便跟踪操作并快速识别问题。

Very often the production support team is the team that accepts the release from the development team. If your organization has this type of hand-off, it is important that your development team works closely with the production support team to make it a smooth transition. Make sure the production support team understands how to use the system’s log files and the messaging and monitoring systems in order to keep track of operations and identify problems quickly.

了解对业务的影响

Understand Impact to Business

每次部署到生产需要中断时,您的客户就无法使用该产品。如果您的产品是网站,这可能会产生巨大的影响。如果您的产品是要下载到 PC 上的独立产品,则影响很小。敏捷团队经常发布以最大化业务价值,小版本发布产生大负面影响的风险较低。与企业合作以按时间段发布时间是常识最大限度地减少中断。尽可能自动化和简化部署流程,以减少停机时间。快速部署过程在短迭代开发期间也很有帮助,我们可能会在一天内部署十几次。

Every time a deployment to production requires an outage, the product is unavailable to your customer. If your product is a website, this may be a huge impact. If your product is an independent product to be downloaded onto a PC, the impact is low. Agile teams release frequently to maximize value to the business, and small releases have a lower risk of a large negative impact. It’s common sense to work with the business to time releases for time periods that minimize disruption. Automate and streamline deployment processes as much as possible to keep downtime windows small. A quick deployment process is also helpful during development in short iterations where we may deploy a dozen times in one day.

新版本应该对客户尽可能透明。发布后需要的紧急发布或补丁越少,您的客户对产品和开发团队的信心就越大。

New releases should be as transparent as possible to the customer. The fewer emergency releases or patches required after a release, the more confidence your customer will have in both the product and the development team.

从每次发布中吸取教训并采取行动使下一次发布更加顺利。获取参与规划的所有角色,例如系统和数据库管理员。评估每个版本并想办法改进下一个版本。

Learn from each release and take actions to make the next one go more smoothly. Get all roles, such as system and database administrators, involved in the planning. Evaluate each release and think of ways to improve the next one.

概括

Summary

本章涵盖了以下几点:

This chapter covered the following points:

图像产品的成功交付不仅仅包括您正在构建的应用程序。规划非软件可交付成果,例如文档、法律声明和培训。

Successful delivery of a product includes more than just the application you are building. Plan the non-software deliverables such as documentation, legal notices, and training.

图像最后的游戏是一个机会,可以对你的产品进行最后的润色。

The end game is an opportunity to put the spit and polish, the final finishing touches, on your product.

图像 其他小组可能负责最终游戏和发布的环境、工具和其他组件。提前与他们协调。

Other groups may be responsible for environments, tools, and other components of the end game and release. Coordinate with them ahead of time.

图像请务必测试数据库更新脚本、数据转换和安装的其他部分。

Be sure to test database update scripts, data conversions, and other parts of the installation.

图像UAT 为客户提供了测试其数据并建立对产品信心的机会。

UAT is an opportunity for customers to test against their data and to build their confidence in the product.

图像根据需要为额外的周期预算时间,例如与外部各方协调测试的后期开发周期。

Budget time for extra cycles as needed, such as post-development cycles to coordinate testing with outside parties.

图像在发布计划期间建立发布验收标准,以便您可以知道何时准备好发布。

Establish release acceptance criteria during release planning so that you can know when you’re ready to release.

图像测试人员通常参与管理发布和测试包装。

Testers often are involved in managing releases and testing the packaging.

图像发布产品时,要考虑整个包装——客户的需求和期望。

When releasing the product, consider the whole package—what the customer needs and expects.

图像从每个版本中学习,并进行调整以改进您的流程。

Learn from each release, and adapt to improve your processes.

第六部分总结

Part VI Summary

第 21 章关键成功因素”中,我们将所有内容放在一起并总结了敏捷测试方法。

In Chapter 21, “Key Success Factors,” we pull things together and summarize the agile approach to testing.

第 21 章关键成功因素

Chapter 21 Key Success Factors

在经历了一个迭代之后,随着敏捷测试人员参与许多活动,我们现在可以挑选出一些关键因素,这些因素可以帮助测试人员在敏捷团队中取得成功,并帮助敏捷团队成功交付高质量的产品。我们认为敏捷测试人员可以提供一些特别的东西。“感染敏捷”的测试人员学习如何应用敏捷实践和原则来帮助他们的整个团队生产出更好的产品。敏捷团队中“被测试感染”的程序员学习如何使用测试来产生更好的工作。角色之间的界限很模糊,但这是一件好事。每个人都专注于质量。

Having traveled through an iteration and beyond, following an agile tester as she engages in many activities, we can now pick out some key factors that help testers succeed on agile teams and help agile teams succeed at delivering a high-quality product. We think agile testers have something special to offer. “Agile-infected” testers learn how to apply agile practices and principles to help their whole team produce a better product. “Test-infected” programmers on agile teams learn how to use testing to produce better work. Lines between roles are blurred, but that’s a good thing. Everyone is focused on quality.

我们从自己的反复试验以及与我们合作的团队中收集了一些适用于敏捷团队和测试人员的关键测试指南。这些指南建立在敏捷测试矩阵、我们学习克服文化和组织障碍的经验、我们在敏捷团队中扮演测试人员角色的冒险经历以及我们弄清楚如何最好地使用测试自动化的经验之上。我们喜欢幸运数字,因此在本章中我们将介绍七个帮助敏捷测试人员取得成功的关键因素。

We have gleaned some critical testing guidelines for agile teams and testers from our own trial and error as well as from teams with which we’ve worked. These guidelines are built on the agile testing matrix, on our experience of learning to overcome cultural and organizational obstacles, our adventures in performing the tester role on agile teams, and our experience of figuring out how best to use test automation. We like lucky numbers, so in this chapter we present seven key factors that help an agile tester succeed.

我们请了一小群正在审阅我们某些章节的人来建议介绍这些成功因素的顺序。结果差异很大,尽管许多人(但不是全部)同意前两名。选择能给您带来最大投资回报的成功因素,并从今天开始努力。

We asked a small group of people who were reviewing some of our chapters to suggest the order in which to present these success factors. The results varied quite a bit, although many (but not all) agreed on the top two. Pick the success factor that will give you the biggest return on investment, and start working on it today.

成功因素 1:使用整个团队的方法

Success Factor 1: Use the Whole-Team Approach

当整个开发团队负责测试和质量时,您将拥有各种各样的技能和经验水平来应对可能出现的任何测试问题。对于一群熟练的程序员来说,测试自动化不是一个大问题。当测试是团队的首要任务,并且任何人都可以报名参加测试任务时,团队就会设计可测试的代码。

When the whole development team takes responsibility for testing and quality, you have a large variety of skill sets and experience levels taking on whatever testing issues might arrive. Test automation isn’t a big problem to a group of skilled programmers. When testing is a team priority, and anyone can sign up for testing tasks, the team designs testable code.

让测试人员真正成为开发团队的一部分意味着为他们提供适应敏捷开发的快节奏所需的支持和培训。他们有时间学习新技能,以便与开发团队和客户团队的成员密切合作。

Making testers truly part of the development team means giving them the support and training they need to adapt to the fast pace of agile development. They have time to acquire new skills in order to collaborate closely with members of both the development and customer teams.

如果您管理一个敏捷团队,请使用第 II 部分组织挑战”中的建议来帮助您的团队采用全团队方法。请记住,质量而非速度是敏捷开发的目标。您的团队需要测试人员帮助客户阐明需求,将需求转化为指导开发的测试,并提供独特的观点来促进可靠产品的交付。确保测试人员可以将他们的技能和专业知识传授给团队的其他成员。确保他们没有被束缚在诸如仅进行手动测试之类的角色中。确保当他们寻求帮助时(这可能需要他们相当大的勇气),他们的团队成员会给予帮助。反之亦然。只要有人需要他们可以提供的帮助,测试人员就应该挺身而出。

If you manage an agile team, use the suggestions in Part II, “Organizational Challenges,” to help your team adopt the whole-team approach. Remember that quality, not speed, is the goal of agile development. Your team needs testers to help customers clarify requirements, turn those into tests that guide development, and provide a unique viewpoint that will promote delivery of a solid product. Make sure the testers can transfer their skills and expertise to the rest of the team. Make sure they aren’t pigeonholed in a role such as only doing manual testing. Make sure that when they ask for help (which may require considerable courage on their part), their team members give it. The reverse is true, too; a tester should step up whenever someone needs assistance that they can provide.

如果您是敏捷团队的测试员,并且正在进行的计划会议和设计讨论中没有您,或者业务用户正在努力单独定义他们的故事和需求,那么是时候站起来和他们谈谈了团队的其他成员。与程序员坐在一起,邀请自己参加会议,并建议让测试人员、程序员和业务专家参与进来,尝试“三的力量”。有用,提供反馈并帮助客户提供示例。让你的问题成为团队的问题,让他们的问题成为你的问题。要求您的队友采用整体团队方法。

If you’re a tester on an agile team, and there are planning meetings and design discussions happening that don’t include you, or the business users are struggling to define their stories and requirements alone, it’s time to get up and go talk to the rest of the team. Sit with the programmers, invite yourself to meetings, and propose trying the “Power of Three” by involving a tester, a programmer, and a business expert. Be useful, giving feedback and helping the customers provide examples. Make your problems the team’s problems, and make their problems yours. Ask your teammates to adopt a whole-team approach.

请参阅第 2 章敏捷测试人员的十项原则”,以了解“三的力量”如何运作的示例。

See Chapter 2, “Ten Principles for Agile Testers,” for an example of how the “Power of Three” works.

成功因素 2:采用敏捷测试思维模式

Success Factor 2: Adopt an Agile Testing Mind-Set

第 2 章敏捷测试人员的十项原则”中,我们告诫敏捷测试人员放弃他们可能带来的任何“质量警察”思维模式。你现在在一个敏捷团队中,程序员测试和测试人员尽他们所能帮助团队交付最好的产品。作为我们在第 2 章强调过,敏捷测试的态度是主动的、有创造力的、对新想法持开放态度并愿意承担任何任务。敏捷测试员不断磨练自己的手艺,随时准备协作,相信自己的直觉,并热衷于帮助团队和业务取得成功。

In Chapter 2, “Ten Principles for Agile Testers,” we cautioned agile testers to lose any “Quality Police” mind-set they might have brought with them. You’re on an agile team now, where programmers test and testers do whatever they can think of to help the team deliver the best possible product. As we emphasized in Chapter 2, an agile testing attitude is proactive, creative, open to new ideas, and willing to take on any task. The agile tester constantly hones her craft, is always ready to collaborate, trusts her instincts, and is passionate about helping the team and the business succeed.

我们并不是说你应该穿上你的超级测试员斗篷去保护世界免受错误的侵害。敏捷团队不允许大自负。您的队友分享您对质量的热情。专注于团队的目标,尽你所能帮助每个人把工作做到最好。

We don’t mean that you should put on your Super Tester cape and go protect the world from bugs. There’s no room for big egos on agile teams. Your teammates share your passion for quality. Focus on the team’s goals and do what you can to help everyone do their best work.

使用敏捷原则和价值观来指导您。始终尝试最简单的方法来满足测试需求。勇于寻求帮助和尝试新想法。专注于提供价值。尽可能直接和经常地沟通。灵活应对变化。请记住,敏捷开发是以人为中心的,我们都应该享受我们的工作。当有疑问时,回到价值观和原则来决定做什么。

Use agile principles and values to guide you. Always try the simplest approach to meeting a testing need. Be courageous in seeking help and experimenting with new ideas. Focus on delivering value. Communicate as directly and as often as possible. Be flexible in responding to change. Remember that agile development is people-centric, and that we should all enjoy our work. When in doubt, go back to the values and principles to decide what to do.

敏捷测试思维模式的一个重要组成部分是不断寻找更好的工作方式的动力。一个成功的敏捷测试员不断地磨练她的手艺。阅读好书、博客和文章以获得新的想法和技能。参加本地用户组会议。参与邮件列表讨论以获得问题或新想法的反馈。如果你的公司不花钱让你参加一个好的会议,那就把你学到的东西写成体验报告,换取免费的会议注册。回馈您的测试和敏捷开发社区也会对您有所帮助。

An important component of the agile testing mind-set is the drive to continually find better ways to work. A successful agile tester constantly polishes her craft. Read good books, blogs, and articles to get new ideas and skills. Attend local user group meetings. Participate in mailing list discussions to get feedback on problems or new ideas. If your company won’t pay for you to attend a good conference, put what you’ve learned into an experience report to exchange for a free conference registration. Giving back to your testing and agile development communities will help you, too.

有关敏捷测试思维的更多信息,请参阅第 2 章,“敏捷测试人员的十项原则”。

See Chapter 2, “Ten Principles for Agile Testers,” for more about the agile testing mindset.

尝试新的实践、工具和技术。鼓励您的团队尝试新方法。短迭代非常适合实验。你可能会失败,但它会很快,你可以尝试别的东西。

Experiment with new practices, tools, and techniques. Encourage your team to try new approaches. Short iterations are ideally suited to experimentation. You might fail, but it’ll be fast, and you can try something else.

如果您管理敏捷测试人员或敏捷团队,请给他们时间学习并为他们所需的培训提供支持。消除障碍,使他们能够做到最好。

If you manage agile testers or an agile team, give them time to learn and provide support for the training they need. Remove obstacles so that they can do their best work.

当您遇到影响测试的问题时,请将这些问题提交给团队。让团队集体讨论克服这些障碍的方法。回顾是讨论问题以及如何解决问题的地方。保持障碍积压并在每次迭代中解决一两个。使用大的可见图表,或它们的虚拟等效物,以确保每个人都知道出现的问题,并且每个人都可以跟踪编码和测试的进度。

When you’re faced with problems that impact testing, bring those problems to the team. Ask the team to brainstorm ways to overcome these obstacles. Retrospectives are one place to talk about issues and how to resolve them. Keep an impediment backlog and address one or two in every iteration. Use big visible charts, or their virtual equivalents, to ensure that everyone is aware of problems that arise and that everyone can track the progress of coding and testing.

成功因素 3:自动化回归测试

Success Factor 3: Automate Regression Testing

如果没有测试自动化,敏捷团队能否成功?也许吧,但我们所知道的成功团队依赖于自动化回归测试。正如我们在本书中经常提到的,如果您将所有时间都花在手动回归测试上,您将永远没有时间进行重要的探索性测试,这些测试将找出代码中潜伏的破坏性行为。

Can an agile team succeed with no test automation? Maybe, but the successful teams that we know rely on automated regression tests. As we’ve said often in this book, if you’re spending all your time doing manual regression testing, you’ll never have time for the important exploratory testing that will ferret out the damaging behaviors lurking in the code.

敏捷开发使用测试来指导开发。为了编写代码使测试通过,您需要一种快速简便的方法来运行测试。如果没有套件提供的短反馈周期和安全网回归,您的团队很快就会陷入技术债务的泥潭,缺陷队列越来越多,速度越来越慢。

Agile development uses tests to guide development. In order to write code to make a test pass, you need a quick and easy way to run the test. Without the short feedback cycle and safety net regression that suites provide, your team will soon become mired in technical debt, with a growing defect queue and ever-slowing velocity.

有关敏捷测试象限的更多信息,请参见第二部分。

See Part II for more on the Agile Testing Quadrants.

自动化回归测试是一项团队工作。整个团队应该为每种类型的测试选择合适的工具。预先考虑测试将使程序员设计代码以便于测试自动化。使用敏捷测试象限和测试自动化金字塔来帮助您有效地自动化不同类型的测试。

Automating regression tests is a team effort. The whole team should choose appropriate tools for each type of test. Thinking about tests up front will let programmers design code for ease of test automation. Use the Agile Testing Quadrants and test automation pyramid to help you automate different types of tests effectively.

有关测试自动化金字塔的更多信息,请参见第 14 章“自动化策略”。

See Chapter 14, “Automation Strategy,” for more on the test automation pyramid.

记住从简单开始。您会惊讶于一些基本的自动化冒烟测试或自动化单元测试可以提供多少价值。

Remember to start simply. You’ll be surprised at how much value some basic automated smoke tests or automated unit tests can provide.

测试自动化是一项团队工作。这也很难,至少一开始是这样。通常需要克服一大堆“痛苦”。如果您管理开发或测试团队,请确保您以时间、培训和动力的形式提供足够的支持。如果您是没有自动化的团队中的测试人员,并且程序员过于疯狂地尝试编写生产代码而无法停下来考虑测试,那么您将面临巨大的挑战。尝试从管理层和团队成员那里获得支持的不同方式,以开始一些微小的自动化工作。

Test automation is a team effort. It’s also hard, at least at first. There’s often a big “hump of pain” to overcome. If you manage a development or testing team, make sure you’re providing enough support in the form of time, training, and motivation. If you’re a tester on a team with no automation, and the programmers are too frantic trying to write production code to stop and think about testing, you have a big challenge ahead of you. Experiment with different ways of getting support from management and from team members to start some tiny automation effort.

有关促进变革的资源,请参阅参考书目。

See the bibliography for resources on promoting change.

成功因素 4:提供和获取反馈

Success Factor 4: Provide and Obtain Feedback

反馈是核心敏捷价值。敏捷的短期迭代旨在提供持续的反馈,以使团队保持在正轨上。测试人员处于独特的位置,可以帮助以自动化测试结果、探索性测试期间的发现以及对系统实际用户的观察的形式提供反馈。

Feedback is a core agile value. The short iterations of agile are designed to provide constant feedback in order to keep the team on track. Testers are in a unique position to help provide feedback in the form of automated test results, discoveries made during exploratory testing, and observations of actual users of the system.

测试人员也需要反馈。你怎么知道你有来自客户的期望行为的正确例子?您怎么知道您编写的测试用例是否正确反映了这些示例?程序员能否通过查看您捕获的示例和您创建的测试来理解要编码的内容?

Testers need feedback too. How do you know that you have the right examples of desired behavior from the customers? How do you know if the test cases you wrote reflected these examples correctly? Can the programmers understand what to code by looking at the examples you’ve captured and the tests you’ve created?

您可以学习的最有价值的技能之一是如何征求对您自己工作的反馈。询问程序员他们是否获得了足够的信息来理解需求,以及这些信息是否指导了他们的编码。询问客户是否觉得他们的质量标准得到满足。在迭代计划会议和回顾中花时间讨论这些问题并提出改进方法。

One of the most valuable skills you can learn is how to ask for feedback on your own work. Ask the programmers if they get enough information to understand requirements and whether that information guides their coding. Ask customers if they feel their quality criteria are being met. Take time in both the iteration planning meetings and retrospectives to talk about these issues and suggest ways to improve.

成功因素 5:建立核心实践基础

Success Factor 5: Build a Foundation of Core Practices

测试行业的一句老话是,“你不能将质量测试到产品中”。当然,敏捷开发也是如此。我们认为不遵循一些基本实践就无法交付高质量的软件。虽然我们将这些视为敏捷实践,但它们比术语“敏捷开发”存在的时间更长,而且它们只是成功软件开发的核心实践。

An old saying in the testing business is, “You can’t test quality into the product.” This is, of course, true of agile development as well. We feel you can’t deliver high-quality software without following some fundamental practices. While we think of these as agile practices, they’ve been around longer than the term “agile development,” and they’re simply core practices of successful software development.

持续集成

Continuous Integration

每个开发团队都需要源代码管理和持续集成才能取得成功。如果您不确切知道,就无法进行有效测试你正在测试什么,如果你没有可以部署的代码,你根本无法测试。所有团队成员每天至少需要检查一次他们的工作。每个集成都必须通过自动构建进行验证,其中包括测试以提供有关软件状态的快速反馈。

Every development team needs source code management and continuous integration to be successful. You can’t test effectively if you don’t know exactly what you’re testing, and you can’t test at all if you have no code you can deploy. All team members need to check in their work at least once a day. Every integration must be verified by an automated build that includes tests to provide rapid feedback about the state of the software.

实施持续集成过程应该是任何软件开发团队的首要任务之一。如果您的团队没有至少一个每日验证的构建,请停止您正在做的事情并开始一个。这很重要。它不一定是完美的开始。如果您要集成一个庞大的系统,那肯定更具挑战性。不过,总的来说,这并不难。有大量优秀的工具,包括开源的和商业的,可用于此目的。

Implementing a continuous integration process should be one of the first priorities of any software development team. If your team doesn’t have at least a daily verified build, stop what you’re doing and get one started. It’s that important. It doesn’t have to be perfect to start with. If you have a huge system to integrate, it’s definitely more challenging. In general, though, it’s not that difficult. There’s a plethora of outstanding tools, both open source and commercial, available for this purpose.

有关持续集成的更多信息,请参阅参考书目。

See the bibliography for more information about continuous integration.

测试环境

Test Environments

如果没有您控制的测试环境,您就无法进行高效测试。您需要知道部署了什么构建,正在使用什么数据库模式,是否有其他人正在更新该模式,以及机器上正在运行哪些其他进程。

You can’t test productively without a test environment that you control. You need to know what build is deployed, what database schema is being used, whether anyone else is updating that schema, and what other processes are running on the machine.

硬件一直在变得越来越便宜,并且有更多可用于测试环境的开源软件可用。您的团队必须进行投资,以便您可以快速有效地进行自动化和手动探索性测试。如果测试环境有问题,请说出来,让团队创造性地解决这个问题。

Hardware is getting less expensive all the time, and more open source software is available that can be used for test environments. Your team must make the investment so that you can effectively conduct automated and manual exploratory tests quickly and efficiently. If there’s a problem with the test environment, speak up and let it be a problem for the team to solve creatively.

管理技术债务

Manage Technical Debt

即使是优秀的软件开发团队,也会感到时间压力,忽视重构或诉诸快速修复和黑客攻击来快速解决问题。随着代码变得更加混乱和难以维护,会出现更多的错误,并且很快团队的速度就会被错误修复和试图理解代码以添加新功能所消耗。您的团队必须不断评估拖累它的技术债务数量,并努力减少和预防它。

Even good software development teams, feeling time pressure, neglect refactoring or resort to quick fixes and hacks to solve a problem quickly. As the code becomes more confusing and hard to maintain, more bugs creep in, and it doesn’t take long before the team’s velocity is consumed by bug fixes and trying to make sense out of the code in order to add new features. Your team must constantly evaluate the amount of technical debt dragging it down and work on reducing and preventing it.

人们经常说,“我们的管理层不会给我们时间把事情做好,我们没有时间重构,而且我们的期限很紧。” 然而,不难找到一个清晰的商业案例来说明不断增长的技术债务给公司带来的损失。有许多方法可以衡量代码和缺陷率,这些方法可以将技术债务转化为对底线的影响。仅仅指出你的减速速度可能就足够了。企业需要他们的软件开发团队保持始终如一的生产力。他们可能不得不缩小所需功能的范围,以便留出足够的时间进行良好的、以测试为导向的代码设计和良好实践,例如持续的小规模重构。

People often say, “Our management won’t give us time to do things right, we don’t have time to refactor, and we’re under tight deadlines.” However, it’s not hard to make a clear business case showing what growing technical debt is costing the company. There are many ways to measure code and defect rates that can translate technical debt into its impact on the bottom line. Merely pointing to your decreasing velocity may be enough. Businesses need their software development teams to remain consistently productive. They may have to reduce the scope of their desired features in order to allow enough time for good, test-guided code design and good practices such as continual small refactoring.

自动回归测试的良好覆盖率是最小化技术债务的关键。如果缺少这些,请在每次迭代中预算时间来构建自动化测试,计划“重构迭代”以升级或添加必要的工具,并编写测试和进行主要的重构工作。在每次迭代中,花时间用测试来指导代码,根据需要重构您正在接触的代码,并在缺少的地方添加自动化测试。增加您的估计以说明这项工作。从长远来看,团队将能够走得更快。

Good coverage from automated regression tests is key to minimizing technical debt. If these are lacking, budget time in each iteration to build up the automated tests, plan a “refactoring iteration” to upgrade or add necessary tools, and write tests and do major refactoring efforts. In every iteration, take the time to guide code with tests, refactor the code you’re touching as needed, and add automated tests where they’re missing. Increase your estimates to account for this work. In the long run, the team will be able to go much faster.

增量工作

Working Incrementally

敏捷团队能够创造优质产品的一个原因是他们的工作规模很小。故事代表几天的工作,每个故事可能被分解成几个薄片或钢丝,并逐步构建。这允许测试每个小块,然后在将小块放在一起时进行增量测试。

One reason agile teams are able to create a quality product is that they work on a small scale. Stories represent a few days of work, and each story may be broken into several thin slices or steel threads and built step-by-step. This allows testing each small piece and then incrementally testing as the pieces are put together.

如果您的团队成员很想同时承担大量功能,请鼓励他们考虑循序渐进的方法。提出问题:“这个故事的核心商业价值是什么?这段代码最基本的路径是什么?接下来会发生什么?建议编写任务卡来编码和测试小块,获得设计的概念证明,并确认您的测试和测试自动化策略。

If your team members are tempted to take on a large chunk of functionality at once, encourage them to look at a stepwise approach. Ask questions: “What’s the central business value in this story? What’s the most basic path through this piece of code? What would come next?” Suggest writing task cards to code and test the small pieces, get a proof of concept for your design, and confirm your test and test automation strategy.

在第 8 章支持团队的面向业务的测试”中阅读有关小块和薄片的更多信息。

Read more about small chunks and thin slices in Chapter 8, “Business-Facing Tests that Support the Team.”

编码和测试是一个过程的一部分

Coding and Testing Are Part of One Process

刚接触敏捷的人经常问敏捷测试人员,“在所有的故事都完成并且可以测试之前,你会做什么?” 经验丰富的敏捷从业者说,“测试人员必须参与整个迭代,整个开发过程。否则是行不通的。”

People who are new to agile often ask agile testers, “What do you do until all the stories are finished and you can test?” Experienced agile practitioners say, “Testers must be involved throughout the whole iteration, the whole development process. Otherwise it doesn’t work.”

测试人员根据客户提供的示例编写测试,以帮助程序员理解故事并开始使用。测试和示例提供了参与软件生产的每个人都能理解的通用语言。随着编码的进行,测试人员和程序员密切合作,他们也都与客户密切合作。程序员向测试人员展示他们编写的功能,而测试人员向程序员展示他们发现的意外行为。随着编码的进行,测试人员编写更多测试,程序员让它们通过,测试人员会进行更多的探索性测试,以了解是否已交付正确的价值。每个敏捷迭代都由几十个持续的、快速的、增量的测试-代码-测试-代码-测试迭代组成。

Testers write tests, based on examples provided by customers, to help programmers understand the story and get started. Tests and examples provide a common language that everyone involved in producing the software understands. Testers and programmers collaborate closely as coding proceeds, and they both also collaborate closely with the customers. Programmers show testers the functionality they’ve written, and testers show programmers the unexpected behaviors they’ve found. Testers write more tests as coding proceeds, programmers make them pass, and testers do more exploratory testing to learn whether the right value has been delivered. Each agile iteration consists of dozens of constant, quick, incremental test-code-test-code-test iterations.

当这种协作和反馈周期受到干扰,并且测试与开发分离时,就会发生不好的事情。如果一个故事在编码后的迭代中进行了测试并发现了错误,程序员必须停止处理新故事,记住代码是如何在上一次迭代的故事中工作的,修复它,然后等待有人测试使固定。软件开发中的事实很少,但我们可以肯定地知道,越早发现错误,修复它们的成本就越低。

When this collaboration and feedback cycle is disturbed, and testing is separated from development, bad things happen. If a story is tested in the iteration after which it was coded and bugs are found, the programmer has to stop working on the new story, remember how the code worked for the last iteration’s story, fix it, and wait for someone to test the fix. There are few facts in software development, but we know for sure that bugs are cheaper to fix the sooner they’re found.

当编码始终由测试指导,并且测试与编码一起进行时,我们更有可能实现行为并提供客户想要的价值。测试是团队的责任。如果您的团队不同意这种观点,请让每个人思考他们对质量的关注、他们交付尽可能最好的产品的愿望,以及他们可以采取哪些步骤来确保团队实现其目标。

When coding is constantly guided by tests, and testing happens alongside coding, we’re much more likely to achieve the behavior and provide the value that the customer wanted. Testing is a team responsibility. If your team doesn’t share this view, ask everyone to think about their focus on quality, their desire to deliver the best possible product, and what steps they can take to ensure that the team achieves its goals.

在第 18 章“编码和测试”中阅读有关编码测试的更多信息。

Read more about coding and testing in Chapter 18, “Coding and Testing.”

实践之间的协同作用

Synergy between Practices

单一的敏捷开发实践(如持续集成)可能会有所作为,但多个敏捷实践的组合大于部分之和。测试驱动设计、集体代码所有权和持续集成一起提供快速反馈,不断改进代码设计和快速交付业务价值的能力。自动化测试很好,但是使用自动化测试来驱动开发,然后通过探索性测试来检测差距或弱点,效果要好很多。

A single agile development practice such as continuous integration can make a difference, but the combination of multiple agile practices is greater than the sum of the parts. Test-driven design, collective code ownership, and continuous integration together deliver rapid feedback, continually improving code design and the ability to deliver business value quickly. Automating tests is good, but using automated tests to drive development, followed up by exploratory testing to detect gaps or weaknesses, is many levels of magnitude better.

有些做法孤立地运作效果不佳。没有自动化测试就不可能进行重构。以迷你瀑布方式进行小版本发布并避免敏捷开发的所有好处是可能的。如果您的现场客户无权做出决定,那么她对团队的价值是有限的。

Some practices don’t work well in isolation. Refactoring is impossible without automated tests. It’s possible to do small releases in a mini-waterfall fashion and avoid all benefits of agile development. If your on-site customer isn’t empowered to make decisions, her value to the team is limited.

敏捷实践旨在相互补充。花时间了解每项实践的目的,考虑充分利用每项实践所需的条件,并就哪些对您的团队有用做出深思熟虑的决定。

Agile practices were designed to complement each other. Take time to understand the purpose of each one, consider what is needed to take full advantage of each practice, and make thoughtful decisions about what works for your team.

成功因素 6:与客户合作

Success Factor 6: Collaborate with Customers

测试人员为敏捷团队贡献的一些最大价值是帮助客户澄清需求并确定需求的优先级,说明需求具有所需行为和用户场景的具体示例,并将这些示例转化为可执行的测试。测试人员使用业务的领域语言和开发团队的技术语言。我们是优秀的促进者和翻译者。

Some of the greatest value that testers contribute to agile teams is helping customers clarify and prioritize requirements, illustrating the requirements with concrete examples of desired behavior and user scenarios, and turning those examples into executable tests. Testers speak the domain language of the business and the technical language of the development team. We make good facilitators and translators.

永远不要妨碍程序员和客户之间的直接交流。鼓励尽可能多的直接交流。使用“三的力量”。当需求被遗漏或被误解时,客户、程序员和测试人员需要共同努力来回答问题。让客户在必要时经常在白板或虚拟等效物前交谈。如果客户分散在园区、国家或全球各地,请使用您能找到的所有工具来加强沟通和协作。电话会议、即时消息和维基不是面对面对话的理想替代品,但它们胜过发送电子邮件或根本不交谈。

Never get in the way of direct communication between programmers and customers. Do encourage as much direct communication as possible. Use the “Power of Three.” When requirements are missed or misunderstood, a customer, programmer, and tester need to work together to get questions answered. Get the customers talking in front of a whiteboard or its virtual equivalent as often as necessary. If customers are scattered around the campus, the country, or the globe, use every tool you can find to enhance communication and collaboration. Teleconferences, instant messages, and wikis aren’t an ideal replacement for face-to-face conversation, but they beat sending emails or not talking at all.

成功因素 7:放眼大局

Success Factor 7: Look at the Big Picture

当然,这是一种概括,但我们发现测试人员倾向于从大局着眼,而且通常是从客户的角度出发。程序员通常必须专注于交付他们现在正在处理的故事,虽然他们可能会使用测试来指导他们,但他们必须专注于需求的技术实现。

This is a generalization, of course, but we’ve found that testers tend to look at the big picture, and usually from a customer point of view. Programmers usually have to focus on delivering the story they’re working on now, and while they may be using tests to guide them, they have to focus on the technical implementation of the requirements.

这种大局观是对团队的巨大贡献。测试驱动的开发如果做得好,可以交付可靠的代码,这些代码在孤立的情况下可能没有缺陷。如果该新功能导致应用程序的某些明显不相关的部分中断怎么办?必须有人考虑对更大系统的影响,并将其引起团队的注意。如果我们忽略了一些会激怒客户的小细节怎么办?新 UI 的编码可能完美无缺,但如果背景颜色使文本难以阅读,最终用户就会注意到这一点。

This big-picture viewpoint is a huge contribution to the team. Test-driven development, done well, delivers solid code that may, in isolation, be free of defects. What if that new feature causes some apparently unrelated part of the application to break? Someone has to consider the impact to the larger system and bring that to the team’s attention. What if we’ve overlooked some little detail that will irritate the customers? The new UI may be flawlessly coded, but if the background color makes the text hard to read, that’s what the end user’s going to notice.

使用敏捷测试象限作为指南,帮助您规划涵盖所有角度的测试。使用测试金字塔的想法来确保测试自动化的良好投资回报率。通过测试指导开发有助于确保您不会错过重要的事情,但这并不完美。使用探索性测试来了解有关应用程序应如何工作以及测试需要采取的方向的更多信息。使用反映真实世界的数据,使您的测试环境尽可能类似于生产环境。努力为负载测试等活动重新创建生产风格的情况。

Use the Agile Testing Quadrants as a guide to help you plan testing that will cover all the angles. Use the test pyramid idea to ensure good ROI from your test automation. Guiding development with tests helps make sure you don’t miss something big, but it’s not perfect. Use exploratory testing to learn more about how the application should work, and what direction your testing needs to take. Make your test environments as similar as possible to production, using data that reflects the real world. Be diligent about re-creating a production-style situation for activities such as load testing.

第三部分解释了如何使用敏捷测试象限。

Part III explains how to use the Agile Testing Quadrants.

团队中的每个人都很容易狭隘地只关注手头的任务或故事。这是一次处理小块功能的缺点。帮助您的团队时不时后退一步,评估您当前的故事如何融入企业的宏伟计划。不断问自己如何才能更好地交付真正的价值。

It’s easy for everyone on the team to narrowly focus only on the task or story at hand. That’s a drawback of working on small chunks of functionality at a time. Help your team take a step back now and then to evaluate how your current stories fit into the grand scheme of the business. Keep asking yourselves how you can do a better job of delivering real value.

概括

Summary

测试和质量是整个团队的责任,但测试人员带来了特殊的观点和独特的技能。作为一名测试人员,您对交付令客户满意的产品的热情将带您度过您和您的团队可能遇到的挫折。不要害怕成为持续改进的推动者。在与客户和开发团队合作时,让敏捷原则和价值观指导您,在每次迭代中增加价值。

Testing and quality are the responsibility of the whole team, but testers bring a special viewpoint and unique skills. As a tester, your passion for delivering a product that delights your customers will carry you through the frustrations you and your team may encounter. Don’t be afraid to be an agent for continual improvement. Let agile principles and values guide you as you work with the customer and development teams, adding value throughout each iteration.

在最后一章中,我们研究了敏捷测试成功的七个关键因素:

In this concluding chapter, we looked at seven key factors for successful agile testing:

1 . 使用整个团队的方法

1. Use the whole-team approach.

2 . 采用敏捷测试思维模式

2. Adopt an agile testing mind-set.

3 . 自动化回归测试

3. Automate regression testing.

4 . 提供并获得反馈

4. Provide and obtain feedback.

5 . 建立核心实践的基础

5. Build a foundation of core practices.

6 . 与客户协作

6. Collaborate with customers.

7 . 看看大图

7. Look at the big picture.

词汇表

Glossary

本词汇表包含作者对本书中所用术语的定义。

This glossary contains the authors’ definitions of terms used throughout this book.

验收测试
Acceptance Test

验收测试是定义每个故事必须交付的商业价值的测试。他们可以验证功能需求或非功能需求,例如性能或可靠性。尽管它们用于帮助指导开发,但它比测试驱动开发中用于代码设计的单元级测试处于更高级别。验收测试是一个广义术语,可能包括面向业务和面向技术的测试。

Acceptance tests are tests that define the business value each story must deliver. They may verify functional requirements or nonfunctional requirements such as performance or reliability. Although they are used to help guide development, it is at a higher level than the unit-level tests used for code design in test-driven development. Acceptance test is a broad term that may include both business-facing and technology-facing tests.

应用程序编程接口 (API)
Application Programming Interface (API)

API 使其他软件能够调用某些功能。API 可能包含支持其他程序发出的请求的函数、过程或类。

APIs enable other software to invoke some piece of functionality. The API may consist of functions, procedures, or classes that support requests made by other programs.

建造
Build

构建是将源代码转换为可部署工件的过程,可以安装该工件以运行应用程序。术语“构建”也指可部署的工件。

A build is the process of converting source code into a deployable artifact that can be installed to run the application. The term “build” also refers to the deployable artifact.

成分
Component

组件是整个系统的较大部分,可以单独部署。例如,在Windows平台上,动态链接库(DLL)作为组件,Java Archives(JAR文件)在Java平台上作为组件,面向服务的体系结构(SOA)使用Web Services作为组件。

A component is a larger part of the overall system that may be separately deployable. For example, on the Windows platform, dynamic linked libraries (DLLs) are used as components, Java Archives (JAR files) are components on the Java platform, and a service-oriented architecture (SOA) uses Web Services as components.

组件测试
Component test

组件测试验证组件的行为。组件测试通过测试对象之间的交互来帮助组件设计。

A component test verifies a component’s behavior. Component tests help with component design by testing interactions between objects.

满意条件
Conditions of satisfaction

满意度条件,也称为满意度条件或业务满意度条件,是客户团队为定义期望的行为而做出的关键假设和决策为给定故事提供的代码。满意度条件是衡量故事结果的标准。他们在与客户就每个故事的高级验收标准进行对话时不断发展。讨论满意度条件有助于识别有风险的假设,并增加团队编写和正确估计完成故事的所有任务的信心。

Conditions of satisfaction, also called satisfaction conditions or conditions of business satisfaction, are key assumptions and decisions made by the customer team to define the desired behavior of the code delivered for a given story. Conditions of satisfaction are criteria by which the outcome of a story can be measured. They evolve during conversations with the customer about high-level acceptance criteria for each story. Discussing conditions of satisfaction helps identify risky assumptions and increases the team’s confidence in writing and correctly estimating all the tasks to complete the story.

上下文驱动测试
Context-driven testing

上下文驱动测试遵循七个原则,第一个原则是任何实践的价值都取决于其上下文。每个新项目和每个新应用程序都可能需要不同的方法来处理项目。所有七种做法都可以在网站www.context-driven-testing.com/上找到。

Context-driven testing follows seven principles, the first being that the value of any practice depends on its context. Every new project and every new application may require different ways of approaching a project. All seven practices can be found on the website www.context-driven-testing.com/.

客户团队
Customer team

客户团队确定业务所需的功能并确定其优先级。在 Scrum 中,这些特性成为史诗或主题,进一步分解为故事并构成产品待办事项列表。客户团队包括开发团队之外的所有利益相关者,例如业务专家、主题专家和最终用户。测试人员和开发人员与客户团队密切合作,为每个故事指定所需行为的示例,并将这些示例转化为测试以指导开发。

The customer team identifies and prioritizes the features needed by the business. In Scrum, these features become epics or themes, which are further broken into stories and comprise the product backlog. Customer teams include all stakeholders outside of the development team, such as business experts, subject-matter experts, and end users. Testers and developers work closely with the customer team to specify examples of desired behavior for each story and turn those examples into tests to guide development.

客户测试
Customer test

客户测试验证客户可见并直接与故事或功能相关的功能片段或片段的行为。术语“面向业务的测试”和“面向客户的测试”指的是与客户测试相同类型的测试。

A customer test verifies the behavior of a slice or piece of functionality that is visible to the customer and related directly back to a story or feature. The terms “business-facing test” and “customer-facing test” refer to the same type of test as customer test.

开发团队
Development team

开发团队是生产客户团队要求的软件的技术团队。参与交付软件的每个人都是开发人员,包括程序员、测试人员、数据库专家、系统管理员、技术作家、架构师、可用性专家和分析师。这个开发团队一起工作来生产软件并为企业创造价值,无论他们是同地团队还是虚拟团队。

The development team is the technical team that produces the software requested by the customer team. Everyone involved in delivering software is a developer, including programmers, testers, database experts, system administrators, technical writers, architects, usability experts, and analysts. This development team works together to produce the software and deliver value to the business, whether they are a co-located team or a virtual team.

史诗
Epic

史诗是客户描述的一项功能或特性,是产品待办事项列表中的一项。一个史诗被分解成相关的故事,然后进行大小和估计。一些团队使用术语“主题”而不是史诗。

An epic is a piece of functionality, or feature, described by the customer and is an item on the product backlog. An epic is broken up into related stories that are then sized and estimated. Some teams use the term “theme” instead of epic.

探索性测试
Exploratory testing

探索性测试是一种交互式测试,它将测试设计与测试执行相结合,并侧重于了解应用程序。有关探索性测试的广泛定义,请参阅第 10 章评估产品的面向业务的测试”。

Exploratory testing is interactive testing that combines test design with test execution and focuses on learning about the application. See Chapter 10, “Business-Facing Tests that Critique the Product,” for an extensive definition of exploratory testing.

假对象
Fake object

假对象用更简单的实现替换了依赖组件的功能。它模拟真实依赖组件的行为,但更易于用于测试目的。

A fake object replaces the functionality of the depended-on component with a simpler implementation. It emulates the behavior of the real depended-on component but is easier to use for testing purposes.

特征
Feature

特性是客户描述的一项功能,是产品待办事项列表中的一项。一个功能被分解成相关的故事,然后进行大小和估计。在敏捷开发中,术语“史诗”或“主题”经常用来代替“特性”。

A feature is a piece of functionality described by the customer and is an item on the product backlog. A feature is broken up into related stories that are then sized and estimated. In agile development, the terms “epic” or “theme” are often used in place of “feature.”

功能测试
Functional test

功能测试验证系统在给定一组输入和/或操作的情况下的预期行为。

Functional tests verify the system’s expected behavior given a set of inputs and/or actions.

绿地
Greenfield

Greenfield 项目是从头开始的新应用程序开发项目,没有现有的代码库。没有任何限制,因此开发团队可以有很多选择。

Greenfield projects are new application development projects starting from scratch with no existing code base. There are no constraints, so development teams have many options open to them.

集成开发环境 (IDE)
Integrated Development Environment (IDE)

集成开发环境或 IDE 是一组支持编程和测试的工具。它通常包括编辑器、编译器或解释器调试器、重构功能和构建自动化工具。IDE 通常支持与源代码控制系统集成,并提供特定于语言的支持以帮助进行代码设计。

An Integrated Development Environment, or IDE, is a set of tools that support programming and testing. It usually includes an editor, compiler or intepreter debugger, refactoring capabilities, and build automation tools. IDEs usually enable integration with a source code control system and provide language-specific support to help with code design.

迭代
Iteration

迭代是一个较短的开发周期,通常为一到四个星期,在该周期结束时可能会交付生产就绪代码。交付整个主题或史诗可能需要多次迭代,每次迭代的长度都相同。有些团队实际上每次迭代都会将代码发布到生产环境,但即使代码未发布,它也已准备好发布。

An iteration is a short development cycle, generally from one to four weeks, at the end of which production-ready code can potentially be delivered. Several iterations, each one the same length, may be needed to deliver an entire theme or epic. Some teams actually release the code to production each iteration, but even if the code isn’t released, it is ready for release.

Java 消息服务 (JMS)
Java Messaging Service (JMS)

Java 消息服务 (JMS) API 是一种消息传递标准,它使基于 Java 2 平台企业版 (J2EE) 的应用程序组件能够创建、发送、接收和读取消息。

The Java Messaging Service (JMS) API is a messaging standard that enables application components based on the Java 2 Platform, Enterprise Edition (J2EE) to create, send, receive, and read messages.

遗留系统
Legacy System

遗留系统是没有任何(或很少)自动回归测试的系统。在遗留代码中引入更改或重构它可能会有风险,因为没有测试来捕获系统行为的意外更改。

A legacy system is one that does not have any (or few) automated regression tests. Introducing changes in legacy code, or refactoring it, might be risky because there are no tests to catch unintended changes in system behavior.

多用途 Internet 邮件扩展 (MIME)
Multipurpose Internet Mail Extensions (MIME)

多用途 Internet 邮件扩展,或 MIME,扩展 Internet 邮件的格式以启用非文本邮件、多部分邮件正文和非 US-ASCII 文本邮件和标题。

Multipurpose Internet Mail Extensions, or MIME, extend the format of Internet mail to enable non-textual messages, multipart message bodies, and non-US-ASCII textual messages and headers.

模拟对象
Mock Object

模拟对象模拟现有对象的响应。它有助于设计和测试对象之间的交互,替换真实组件,以便测试可以验证其间接输出。

A mock object simulates the responses of an existing object. It helps with designing and testing interactions between objects, replacing a real component so that a test can verify its indirect outputs.

产品积压
Product Backlog

Product Backlog 是一个 Scrum 术语,指的是产品所需所有功能的优先主列表。随着组织考虑他们可能需要的新功能,这种积压会随着时间的推移而增加。

Product Backlog is a Scrum term for the prioritized master list of all functionality desired in the product. This backlog grows over time as the organization thinks of new features they may need.

产品拥有者
Product Owner

产品负责人是一个 Scrum 术语,指的是负责确定产品待办事项或故事优先级的人。他或她通常是营销人员或参与开发的关键业务专家。

Product Owner is a Scrum term for the person responsible for prioritizing the product backlog, or stories. He or she is typically someone from a marketing role or a key business expert involved with development.

质量保证 (QA) 团队
Quality Assurance (QA) Team

质量保证或 QA 可以定义为为确保符合质量标准而采取的行动。在软件开发中,术语“QA 团队”通常用于指代进行软件测试的团队。测试团队(参见测试团队)为涉众提供与软件产品质量相关的信息。他们执行活动以了解被测系统的行为方式并验证其行为是否符合预期。在敏捷开发中,这些活动与开发活动完全集成。测试人员通常是开发团队的一部分,与其他参与软件开发的人一样。

Quality Assurance, or QA, can be defined as actions taken to ensure compliance with a quality standard. In software development, the term “QA Team” is often used to refer to the team that does software testing. Test teams (see Test Team) provide stakeholders with information related to the quality of the software product. They perform activities to learn how the system under test should behave and verify that it behaves as expected. In agile development, these activities are fully integrated with development activities. Testers are often part of the development team along with everyone else involved in developing the software.

生产代码
Production Code

生产代码是用于或将用于生产的系统代码,与为测试它而编写的代码不同。测试代码调用或操作生产代码以验证其行为。

Production code is the code for the system that is, or will be, used in production, as distinguished from the code that is written to test it. Test code invokes or operates on production code to verify its behavior.

重构
Refactoring

重构是在不改变其功能的情况下更改代码,以使其更易于维护、更易于阅读、更易于测试或更易于扩展。

Refactoring is changing code, without changing its functionality, to make it more maintainable, easier to read, easier to test, or easier to extend.

回归测试
Regression Test

回归测试验证被测系统的行为没有改变。回归测试通常编写为单元测试以驱动编码或验收测试以定义所需的系统行为。一旦测试通过,它们就会成为回归测试套件的一部分,以防止引入意外更改。回归测试应该是自动化的,以确保持续的反馈。

A regression test verifies that the behavior of the system under test hasn’t changed. Regression tests are usually written as unit tests to drive coding or acceptance tests to define desired system behavior. Once the tests pass, they become part of a regression test suite, to guard against unintended changes being introduced. Regression tests should be automated to ensure continual feedback.

发布候选
Release Candidate

候选发布版是可能发布到生产环境的产品版本或构建。候选发布版可能会接受进一步测试,或者通过文档或其他材料进行扩充。

A release candidate is a version or build of the product that can potentially be released to production. The release candidate may undergo further testing or be augmented with documentation or other materials.

投资回报率 (ROI)
Return On Investment (ROI)

投资回报率或 ROI 是从金融投资领域借用的术语,是衡量投资效率的指标。投资回报率可以用不同的方式计算,但它基本上是投资收益与投资成本之间的差值除以投资成本。在测试中,ROI 是从测试活动(例如自动化测试)中获得的收益,与生成和维护该测试或活动的成本进行权衡。

Return on investment, or ROI, is a term borrowed from the world of financial investments and is a measure of the efficiency of an investment. ROI can be calculated in different ways, but it’s basically the difference between the gain from an investment and the cost of that investment, divided by the cost of that investment. In testing, ROI is the benefit gained from a testing activity such as automating a test, weighed against the cost of producing and maintaining that test or activity.

肥皂
SOAP

SOAP 是一种通过网络交换基于 XML 的消息的协议,通常使用 HTTP/HTTPS。它构成了 Web 服务协议栈的基础层,提供了一个可以构建抽象层的基本消息传递框架。一种常见的 SOAP 消息传递模式是远程过程调用 (RPC) 模式,其中客户端网络节点向服务器节点发送请求消息,服务器立即向客户端发送响应。

SOAP is a protocol for exchanging XML-based messages over networks, normally using HTTP/HTTPS. It forms the foundation layer of the web services protocol stack, providing a basic messaging framework upon which abstract layers can be built. A common SOAP messaging pattern is the Remote Procedure Call (RPC) pattern, in which the client network node sends a request message to the server node, and the server immediately sends a response to the client.

故事
Story

用户故事是从用户的角度讲述的对用户或客户团队有价值的功能的简短描述。故事传统上写在索引卡上。该卡片通常包含对该功能的单行描述。例如,“作为购物者,我可以将商品放入购物车,以便稍后结账”是一个故事。卡片只能与客户团队和开发团队之间的后续对话结合使用,并通过编写和运行测试验证故事是否已实现。

A user story is a short description of functionality told from the perspective of the user that is valuable to either the user or the customer team. Stories are traditionally written on index cards. The card typically contains a one-line description of the feature. For example, “As a shopper, I can put items in my shopping cart so that I can check out with them later” is a story. Cards are only useable in combination with subsequent conversations between the customer team and the development team and some verification that the story has been implemented through writing and running tests.

故事测试
Story Test

故事测试定义故事交付的代码的预期行为。故事测试可能是面向业务的,指定功能需求,也可能是面向技术的,例如安全或性能测试。这些测试用于指导开发以及验证交付的代码。大多数敏捷实践者使用术语“故事测试”作为“验收测试”的同义词,尽管术语“验收测试”可能用于验证比故事更高级别的行为的测试。

A story test defines expected behavior for the code to be delivered by the story. Story tests may be business-facing, specifying the functional requirements, or technology-facing, such as security or performance tests. These tests are used to guide development as well as to verify the delivered code. Most agile practitioners use the term “story test” synonymously with “acceptance test,” although the term “acceptance test” might be used for tests that verify behavior at a higher level than one story.

故事板
Story Board

故事板,也称为任务板,用于跟踪团队在迭代期间所做的工作。为每个故事编写任务卡,颜色可能与任务类型相协调。这些卡片连同某种视觉提示提供了一种简单的机制来查看迭代进度的当前状态。它可以使用列或卡片上的不同颜色贴纸代表不同的状态,例如“待办事项”、“进行中的工作”、“验证”和“完成”。故事板可以是墙上的物理板或虚拟在线板。

The story board, also called the task board, is used to track the work the team does during an iteration. Task cards, which may be color-coordinated for the type of task, are written for each story. These cards, along with a visual cue of some kind, provide an easy mechanism for seeing the current status of an iteration’s progress. It may use columns or different colored stickers on cards for different states such as “To do,” “Work in Progress,” “Verify,” and “Done.” The story board might be a physical board on a wall or a virtual online board.

任务
Task

任务是完成故事所需的工作。任务可能是实现故事的一小部分所需的操作,也可能是构建一些基础设施或测试包含多个故事的内容。通常它应该代表一天或更少的工作。

Tasks are pieces of work needed to finish a story. A task might be action needed to implement a small piece of a story, or it might be for building a bit of infrastructure, or testing that encompasses more than one story. Generally it should represent a day or less of work.

技术债务
Technical Debt

Ward Cunningham 首先介绍了这个比喻。当团队在不使用 TDD、持续集成和重构等良好实践的情况下生产软件时,可能会产生技术债务。与金融债务一样,技术债务会产生利息,这将使团队在以后付出更多代价。有时这笔债可能是值得的,比如利用一个突如其来的商机。不过,通常情况下,技术债务会加剧并减慢团队的速度。每次迭代中产生的商业价值越来越少,因为代码缺乏自动回归测试的安全网,或者变得难以理解和维护。

Ward Cunningham first introduced this metaphor. When a team produces software without using good practices such as TDD, continuous integration, and refactoring, it may incur technical debt. Like financial debt, technical debt accrues interest that will cost the team more at a later date. Sometimes this debt may be worthwhile, such as to take advantage of a sudden business opportunity. Usually, though, technical debt compounds and slows the team’s velocity. Less and less business value can be produced in each iteration because the code lacks a safety net of automated regression tests or has become difficult to understand and maintain.

测试替身
Test Double

测试替身是为了运行测试的明确目的而安装在真实组件位置的任何对象或组件。测试替身包括虚拟对象、模拟对象、测试存根和假对象。

A test double is any object or component that’s installed in place of the real component for the express purpose of running a test. Test doubles include dummy objects, mock objects, test stubs, and fake objects.

测试驱动开发 (TDD)
Test-Driven Development (TDD)

在测试驱动开发中,程序员先编写并自动化一个小的单元测试,然后再编写使测试通过的小代码段。生产代码一次只能进行一个测试。

In test-driven development, the programmer writes and automates a small unit test before writing the small piece of code that will make the test pass. The production code is made to work one test at a time.

测试先行开发
Test-First Development

在测试先行的开发中,测试是在相应的生产代码之前编写的,但代码不一定一次只能运行一个测试。客户或故事测试可用于测试先行开发以及单元测试。

In test-first development, tests are written in advance of the corresponding production code, but the code is not necessarily made to work one test at a time. Customer or story tests may be used in test-first development as well as unit tests.

测试存根
Test Stub

测试存根是一个对象,它将被测系统所需的真实组件替换为特定于测试的对象,该对象将所需的间接输入提供给被测系统。这使测试能够独立于其他组件来验证逻辑。

A test stub is an object that replaces a real component needed by the system under test with a test-specific object that feeds desired indirect inputs into the system under test. This enables the test to verify logic independently of the other components.

测试团队
Test Team

测试团队执行的活动有助于定义并随后验证被测系统的预期行为。测试团队向利益相关者提供有关系统外部质量、可能存在的风险以及潜在的风险缓解策略的信息。在敏捷开发中,这些活动与发展活动充分结合。测试人员通常是开发团队的一部分,与其他参与软件开发的人一样。

A test team performs activities that help define and subsequently verify the desired behavior of the system under test. The test team provides information to the stakeholders about the external quality of the system, the risks that may be present, and potential risk mitigation strategies. In agile development, these activities are fully integrated with development activities. Testers are often part of the development team along with everyone else involved in developing the software.

测试仪
Tester

测试人员向利益相关者提供有关正在开发的软件的信息。测试人员帮助客户定义功能和非功能需求以及质量标准,并帮助将这些转化为指导开发和验证所需行为的测试。测试人员执行与交付高质量软件相关的各种活动,例如测试自动化和探索性测试。在敏捷开发中,开发团队中的每个人都执行测试活动。自称为测试人员的团队成员与开发人员和客户团队的其他成员密切合作。

A tester provides information to stakeholders about the software being developed. A tester helps customers define functional and nonfunctional requirements and quality criteria, and helps turn these into tests that guide development and verify desired behavior. Testers perform a wide variety of activities related to delivering high-quality software, such as test automation and exploratory testing. In agile development, everyone on the development team performs testing activities. Team members who identify themselves as testers work closely with other members of both the developer and customer teams.

主题
Theme

主题与史诗或故事片相同。它是客户描述并放置在产品待办事项列表中的一项功能,将被分解成大小和估计的故事。

A theme is the same as an epic or feature. It is a piece of functionality described by the customer and placed in the product backlog to be broken up into stories that are sized and estimated.

单元测试
Unit Test

单元测试验证整个系统的一小部分的行为。它可能小到单个对象或方法,是一个或多个设计决策的结果。

A unit test verifies the behavior of a small part of the overall system. It may be as small as a single object or method that is a consequence of one or more design decisions.

速度
Velocity

开发团队的速度是它在每次迭代中交付的价值量,以故事点、理想天数或小时数衡量。通常,速度中只包含已完成的故事。Velocity 有助于企业规划未来的功能和版本。敏捷团队使用上一次迭代的速度来帮助确定他们在下一次迭代中可以承担的工作量。

A development team’s velocity is the amount of value it delivers in each iteration, measured in story points, ideal days, or hours. Generally, only completed stories are included in the velocity. Velocity is helpful to the business in planning for future features and releases. Agile teams use their velocity for the previous iteration to help determine the amount of work they can take on in the next iteration.

网络服务描述语言 (WSDL)
Web Service Description Language (WSDL)

Web 服务描述语言 (WDSL) 是一种 XML 格式,用于将网络服务描述为一组端点,这些端点对包含面向文档或面向过程的信息的消息进行操作。

Web Service Description Language (WDSL) is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information.

参考书目

Bibliography

书籍、文章、论文和博客文章

Books, Articles, Papers, and Blog Postings

敏捷联盟。“敏捷宣言背后的原则”,www.agilemanifesto.org/principles.html,2001 年。

Agile Alliance. “Principles Behind the Agile Manifesto,” www.agilemanifesto.org/principles.html, 2001.

Alles、Micah、David Crosby、Carl Erickson、Brian Harleton、Michael Marsiglia、Greg Pattison 和 Curt Stienstra。“Presenter First:为测试驱动开发组织复杂的 GUI 应用程序”,Agile 2006,明尼苏达州明尼阿波利斯,2006 年 7 月。

Alles, Micah, David Crosby, Carl Erickson, Brian Harleton, Michael Marsiglia, Greg Pattison, and Curt Stienstra. “Presenter First: Organizing Complex GUI Applications for Test-Driven Development,” Agile 2006, Minneapolis, MN, July 2006.

安布勒,斯科特。敏捷数据库技术:敏捷软件开发人员的有效策略,Wiley,2003 年。

Ambler, Scott. Agile Database Techniques: Effective Strategies for the Agile Software Developer, Wiley, 2003.

阿斯特尔斯,大卫。测试驱动开发:实用指南,Prentice Hall,2003 年。

Astels, David. Test-Driven Development: A Practical Guide, Prentice Hall, 2003.

巴赫,詹姆斯。“解释性测试”,www.satisfice.com/articles/ et-article.pdf,2003 年。

Bach, James. “Exploratory Testing Explained,” www.satisfice.com/articles/et-article.pdf, 2003.

巴赫,乔纳森。“基于会话的测试管理”,软件测试和质量工程杂志,2000 年 11 月,www.satisfice.com/ articles/sbtm.pdf 。

Bach, Jonathan. “Session-Based Test Management,” Software Testing and Quality Engineering Magazine, November, 2000, www.satisfice.com/articles/sbtm.pdf.

贝克,肯特。极限编程解释:拥抱变化,Addison-Wesley,2000 年。

Beck, Kent. Extreme Programming Explained: Embrace Change, Addison-Wesley, 2000.

贝克,肯特和安德烈斯,辛西娅。极限编程解释:拥抱变化。第 2 版,Addison-Wesley,2004 年。

Beck, Kent, and Andres, Cynthia. Extreme Programming Explained: Embrace Change. 2nd Edition, Addison-Wesley, 2004.

Berczuk、Stephen 和 Brad Appleton。软件配置管理模式:有效的团队合作,实用集成,Addison-Wesley,2003 年。

Berczuk, Stephen and Brad Appleton. Software Configuration Management Patterns: Effective Teamwork, Practical Integration, Addison-Wesley, 2003.

博尔顿,迈克尔。“不使用地图进行测试”,Better Software,2005 年 1 月,www.developsense.com /articles/Testing%20Without%20A%20Map.pdf 。

Bolton, Michael. “Testing Without a Map,” Better Software, January 2005, www.developsense.com/articles/Testing%20Without%20A%20Map.pdf.

Bos、Erik 和 Christ Vriens。“敏捷 CMM”,摘自极限编程和敏捷方法 – XP/Agile Universe 2004,第四届极限编程和敏捷方法会议,加拿大卡尔加里,2004 年 8 月 15 日至 18 日,会议记录,编辑。Carmen Zannier、Hakan Erdogmus、Lowell Lindstrom,第 129–138 页,Springer,2004 年。

Bos, Erik and Christ Vriens. “An Agile CMM,” in Extreme Programming and Agile Methods–XP/Agile Universe 2004, 4th Conference on Extreme Programming and Agile Methods, Calgary, Canada, August 15–18, 2004, Proceedings, ed. Carmen Zannier, Hakan Erdogmus, Lowell Lindstrom, pp. 129–138, Springer, 2004.

Boutelle,乔纳森。“敏捷开发的可用性测试”,www.jonathanboutelle.com/ mt/archives/2005/08/usability_testi_1.html,2005 年。

Boutelle, Jonathan. “Usability Testing for Agile Development,” www.jonathanboutelle.com/mt/archives/2005/08/usability_testi_1.html, 2005.

布朗,泰特斯。“(缺乏)测试死亡螺旋”,http://ivory.idyll.org/blog/mar-08/software-quality-death-spiral.html,2008年。

Brown, Titus. “The (Lack of) Testing Death Spiral,” http://ivory.idyll.org/blog/mar-08/software-quality-death-spiral.html, 2008.

布瓦尔达,汉斯。“肥皂剧测试”,Better Software Magazine,2004 年 2 月,www.logigear.com/ resources/articles_lg/soap_opera_testing.asp 。

Buwalda, Hans. “Soap Opera Testing,” Better Software Magazine, February 2004, www.logigear.com/resources/articles_lg/soap_opera_testing.asp.

克拉克,迈克。实用项目自动化:如何构建、部署和监控 Java 应用程序,实用程序员,2004 年。

Clark, Mike. Pragmatic Project Automation: How to Build, Deploy and Monitor Java Apps, The Pragmatic Programmers, 2004.

科恩,迈克。应用于敏捷软件开发的用户故事,Addison-Wesley,2004 年。

Cohn, Mike. User Stories Applied for Agile Software Development, Addison-Wesley, 2004.

科恩,迈克。敏捷估算和规划,Prentice Hall,2005 年。

Cohn, Mike. Agile Estimating and Planning, Prentice Hall, 2005.

Crispin、Lisa 和 Tip House。测试极限编程,Addison-Wesley 2002。

Crispin, Lisa and Tip House. Testing Extreme Programming, Addison-Wesley 2002.

克里斯平,丽莎。文章“雇用敏捷测试人员”、“Web 测试工具的敏捷工具选择策略”、“提高软件质量:测试驱动开发如何影响软件质量”,http: //lisa.crispin.home.att.net

Crispin, Lisa. Articles “Hiring an Agile Tester,” “An Agile Tool Selection Strategy for Web Testing Tools,” “Driving Software Quality: How Test-Driven Development Impacts Software Quality,” http://lisa.crispin.home.att.net.

DeMarco、Tom 和 Timothy Lister。管理软件项目的风险,Dorset House,2003 年。

DeMarco, Tom and Timothy Lister. Managing Risk on Software Projects, Dorset House, 2003.

德比,以斯帖和拉森,戴安娜。敏捷回顾:让优秀的团队更伟大,Pragmatic Bookshelf,2006 年。

Derby, Esther and Larsen, Diana. Agile Retrospectives: Making Good Teams Great, Pragmatic Bookshelf, 2006.

德比,以斯帖和罗斯曼,约翰娜。闭门造车:伟大管理的秘密,实用书架,2006 年。

Derby, Esther and Rothman, Johanna. Behind Closed Doors: Secrets of Great Management, Pragmatic Bookshelf, 2006.

德索萨,肯。“穿着开发人员外衣的测试员”博客,http://kendesouza.blogspot.com

De Souza, Ken. “A tester in developer’s clothes” blog, http://kendesouza.blogspot.com.

Dustin、Elfriede、Chris Wysopal、Lucas Nelson 和 Dino Dia Zovi。软件安全测试的艺术:识别软件安全缺陷,赛门铁克出版社,2006 年。

Dustin, Elfriede, Chris Wysopal, Lucas Nelson, and Dino Dia Zovi. The Art of Software Security Testing: Identifying Software Security Flaws, Symantec Press, 2006.

达斯汀,埃尔弗里德。“Teamwork Tacks the Quality Goal”,软件测试与性能,第 2 卷,第 200 期,2005 年 3 月。

Dustin, Elfriede. “Teamwork Tackles the Quality Goal,” Software Test & Performance, Volume 2, Issue 200, March 2005.

Duvall、Paul、Steve Matyas 和 Andrew Glover。持续集成:提高软件质量和降低风险,Addison-Wesley,2007 年。

Duvall, Paul, Steve Matyas, and Andrew Glover. Continuous Integration: Improving Software Quality and Reducing Risk, Addison-Wesley, 2007.

埃克斯坦,尤塔。大型敏捷软件开发:深入研究,Dorset House,2004 年。

Eckstein, Jutta. Agile Software Development in the Large: Diving Into the Deep, Dorset House, 2004.

埃文斯,埃里克。领域驱动设计:应对软件核心的复杂性,Addison-Wesley,2003 年。

Evans, Eric. Domain-Driven Design: Tackling Complexity in the Heart of Software, Addison-Wesley, 2003.

羽毛,迈克尔。有效地使用遗留代码,Prentice Hall,2004 年。

Feathers, Michael. Working Effectively with Legacy Code, Prentice Hall, 2004.

弗里曼、史蒂夫和纳特普赖斯。“模拟对象”,www.mockobjects.com

Freeman, Steve and Nat Pryce. “Mock Objects,” www.mockobjects.com.

福勒,马丁。“持续集成”,http://martinfowler.com/articles/continuousIntegration.html,2006年。

Fowler, Martin. “Continuous Integration,” http://martinfowler.com/articles/continuousIntegration.html, 2006.

福勒,马丁。“StranglerApplication”,www.martinfowler.com/bliki /StranglerApplication.html,2004 年。

Fowler, Martin. “StranglerApplication,” www.martinfowler.com/bliki/StranglerApplication.html, 2004.

Fowler, Martin,“TechnicalDebt”, http: //martinfowler.com/bliki/TechnicalDebt.html,2003年。

Fowler, Martin, “TechnicalDebt,” http://martinfowler.com/bliki/TechnicalDebt.html, 2003.

Gårtner, Markus,个人通讯,2008 年。有关更多有趣的信息,请访问他的博客:http: //blog.shino.de

Gårtner, Markus, personal communication, 2008. For more interesting information, visit his blog at: http://blog.shino.de.

盖伦,罗伯特。Software Endgames: Eliminating Defects, Controlling Change, and the Countdown to On-Time Delivery,Dorset House,2005 年。

Galen, Robert. Software Endgames: Eliminating Defects, Controlling Change, and the Countdown to On-Time Delivery, Dorset House, 2005.

Ghiorghiu,格里格。“性能与负载与压力测试”,http://agiletesting.blogspot.com/2005/02/performance-vs-load-vs-stress-testing.html,2005年。

Ghiorghiu, Grig. “Performance vs. load vs. stress testing,” http://agiletesting.blogspot.com/2005/02/performance-vs-load-vs-stress-testing.html, 2005.

Ghirghiu,格里格。“敏捷测试”博客,http://agiletesting.blogspot.com

Ghirghiu, Grig. “Agile Testing” blog, http://agiletesting.blogspot.com.

夏甲,乔恩。软件测试论文,www.swtesting.com/hagar_papers_index.html

Hagar, Jon. Software Testing Papers, www.swtesting.com/hagar_papers_index.html.

亨德里克森,伊丽莎白。“测试开发人员,开发测试人员”,http://testobsessed.com/2007/01/17/tester-developers-developer-testers/,2007年。

Hendrickson, Elisabeth. “Tester Developers, Developer Testers,” http://testobsessed.com/2007/01/17/tester-developers-developer-testers/, 2007.

亨德里克森,伊丽莎白。“测试启发式备忘单”,http://testobsessed.com/wordpress/wp-content/uploads/2007/02/testheuristicscheatsheetv1.pdf,2007年。

Hendrickson, Elisabeth. “Test Heuristics Cheat Sheet,” http://testobsessed.com/wordpress/wp-content/uploads/2007/02/testheuristicscheatsheetv1.pdf, 2007.

亨德里克森,伊丽莎白。“敏捷友好型测试自动化工具/框架”,http://testobsessed.com/2008/04/29/agile-friendly-test-automation-toolsframeworks,2008年。

Hendrickson, Elisabeth. “Agile-Friendly Test Automation Tools/Frameworks,” http://testobsessed.com/2008/04/29/agile-friendly-test-automation-toolsframeworks, 2008.

海史密斯,吉姆。敏捷项目管理:创造创新产品,Addison-Wesley,2004 年。

Highsmith, Jim. Agile Project Management: Creating Innovative Products, Addison-Wesley, 2004.

亨特、安德鲁和大卫·托马斯。务实的程序员:从熟练工到大师,Addison-Wesley,1999 年。

Hunt, Andrew and David Thomas. The Pragmatic Programmer: From Journeyman to Master, Addison-Wesley, 1999.

Kaner、Cem、James Bach 和 Bret Pettichord。Lessons Learned in Software Testing,Wiley,2001 年。

Kaner, Cem, James Bach, and Bret Pettichord. Lessons Learned in Software Testing, Wiley, 2001.

克思,诺曼。项目回顾:团队审查手册,Dorset House,2001 年。

Kerth, Norman. Project Retrospectives: A Handbook for Team Reviews, Dorset House, 2001.

克尼伯格,亨里克。“如何赶上测试自动化”,http://blog.crisp.se/henrikkniberg/2008/01/03/1199386980000.html,2008年。

Kniberg, Henrik. “How to Catch Up on Test Automation,” http://blog.crisp.se/henrikkniberg/2008/01/03/1199386980000.html, 2008.

克尼伯格,亨里克。Scrum 和 XP from the Trenches,Lulu.com,2007 年。

Kniberg, Henrik. Scrum and XP from the Trenches, Lulu.com, 2007.

Koenig、Dierk、Andrew Glover、Paul King、Guillaume Laforge 和 Jon Skeet。Groovy in Action,曼宁出版社,2007 年。

Koenig, Dierk, Andrew Glover, Paul King, Guillaume Laforge, and Jon Skeet. Groovy in Action, Manning Publications, 2007.

Kohl, Jonathan。“人与机器”,Better Software杂志,2007 年 12 月。

Kohl, Jonathan.“Man and Machine,” Better Software magazine, December 2007.

科尔,乔纳森。博客和文章,www.kohl.ca/

Kohl, Jonathan. Blog and articles, www.kohl.ca/.

卢维翁,克里斯托夫。博客,www.runningagile.com

Louvion, Christophe. Blog, www.runningagile.com.

曼斯、玛丽·林恩和琳达·瑞辛。无所畏惧的变革:引入新想法的模式,Addison-Wesley,2004 年。

Manns, Mary Lynn and Linda Rising. Fearless Change: Patterns for Introducing New Ideas, Addison-Wesley, 2004.

马里克,布赖恩。Everyday Scripting with Ruby: For Teams, Testers and You , Pragmatic Bookshelf, 2007.

Marick, Brian. Everyday Scripting with Ruby: For Teams, Testers and You, Pragmatic Bookshelf, 2007.

Marick, Brian,“我的敏捷测试项目”,www.exampler.com/ old-blog/2003/08/21/,2003 年。

Marick, Brian, “My Agile Testing Project,” www.exampler.com/old-blog/2003/08/21/, 2003.

马里克,布赖恩。“面向业务的 TDD 的替代方案”,www.exampler.com/blog /category/aa-ftt,2008 年。

Marick, Brian. “An Alternative to Business-Facing TDD,” www.exampler.com/blog/category/aa-ftt, 2008.

马里克,布赖恩。关于敏捷测试的博客和文章,http://exampler.com

Marick, Brian. Blog and articles on agile testing, http://exampler.com.

马尔卡诺,安东尼。博客,www.testingreflections.com

Marcano, Antony. Blog, www.testingreflections.com.

梅萨罗斯,杰拉德。XUnit 测试模式:重构测试代码,Addison-Wesley,2007 年。

Meszaros, Gerard. XUnit Test Patterns: Refactoring Test Code, Addison-Wesley, 2007.

Meszaros、Gerard 和 Janice Aston。“将可用性测试添加到敏捷项目”,Agile 2006,明尼苏达州明尼阿波利斯,2006 年,http://papers.gerardmeszaros.com/AgileUsabilityPaper.pdf

Meszaros, Gerard and Janice Aston. “Adding Usability Testing to an Agile Project,” Agile 2006, Minneapolis, MN, 2006, http://papers.gerardmeszaros.com/AgileUsabilityPaper.pdf.

Meszaros、Gerard、Ralph Bohnet 和 Jennitta Andrea。“使用记录和回放进行敏捷回归测试”,XP/Agile Universe 2003,路易斯安那州新奥尔良,2003 年,http://agileregressiontestpaper.gerardmeszaros.com

Meszaros, Gerard, Ralph Bohnet, and Jennitta Andrea. “Agile Regression Testing Using Record & Playback,” XP/Agile Universe 2003, New Orleans, LA, 2003, http://agileregressiontestpaper.gerardmeszaros.com.

梅萨罗斯,杰拉德。“使用故事类型拆分臃肿的 XP 故事”,http://storyotypespaper.gerardmeszaros.com

Meszaros, Gerard. “Using Storyotypes to Split Bloated XP Stories,” http://storyotypespaper.gerardmeszaros.com.

马格里奇、里克和沃德坎宁安。适合开发软件:集成测试框架,Prentice Hall,2005 年。

Mugridge, Rick and Ward Cunningham. Fit for Developing Software: Framework for Integrated Tests, Prentice Hall, 2005.

纽柯克、詹姆斯和阿列克谢·沃龙佐夫。Microsoft .NET 中的测试驱动开发,Microsoft Professional,2004。

Newkirk, James and Alexei Vorontsov. Test-Driven Development in Microsoft .NET, Microsoft Professional, 2004.

尼尔森,雅各布。“可用性会议的时间预算”,www.useit.com/alertbox /usability_sessions.html,2005 年。

Nielsen, Jakob. “Time Budgets for Usability Sessions,” www.useit.com/alertbox/usability_sessions.html, 2005.

北方,丹。“BDD 简介”,http://dannorth.net/introducing-bdd,2006年。

North, Dan. “Introducing BDD,” http://dannorth.net/introducing-bdd, 2006.

Patterson、Kerry、Joseph Gernny、Ben McMillan、Al Switzler 和 Stephen R. Covey。关键对话:利害攸关时的谈话工具,McGraw-Hill,2002 年。

Patterson, Kerry, Joseph Gernny, Ben McMillan, Al Switzler and Stephen R. Covey. Crucial Conversations: Tools for Talking when the Stakes are High, McGraw-Hill, 2002.

巴顿,杰夫。“编码前测试软件”,StickyMinds.com,2006 年 8 月,www.stickyminds.com/ sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=11104 。

Patton, Jeff. “Test Software Before You Code,” StickyMinds.com, August 2006, www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=11104.

巴顿,杰夫。“整体敏捷产品设计和开发”,www.agileproductdesign.com/blog /agile_product_development.html,2006 年。

Patton, Jeff. “Holistic Agile Product Design and Development,” www.agileproductdesign.com/blog/agile_product_development.html, 2006.

波尔斯,安迪。“完美的客户”,www.pols.co.uk/archives /category/testing,2008 年。

Pols, Andy. “The Perfect Customer,” www.pols.co.uk/archives/category/testing, 2008.

佩蒂索德,布雷特。“自制测试自动化”,www.io.com/~ wazmo/papers/homebrew_test_automation_200409.pdf,2004 年。

Pettichord, Bret. “Homebrew Test Automation,” www.io.com/~wazmo/papers/homebrew_test_automation_200409.pdf, 2004.

佩蒂索德,布雷特。“测试自动化成功的七个步骤”,www.io.com/~wazmo /papers/seven_steps.html,2001 年。

Pettichord, Bret. “Seven Steps to Test Automation Success,” www.io.com/~wazmo/papers/seven_steps.html, 2001.

Poppendieck、Mary 和 Tom Poppendieck。实施精益软件开发:从概念到现金,Addison-Wesley,2006 年。

Poppendieck, Mary and Tom Poppendieck. Implementing Lean Software Development: From Concept to Cash, Addison-Wesley, 2006.

Poppendieck、Mary 和 Tom Poppendieck。精益软件开发:敏捷工具包,Addison-Wesley,2003 年。

Poppendieck, Mary and Tom Poppendieck. Lean Software Development: An Agile Toolkit, Addison-Wesley, 2003.

Rainsberger,JB JUnit Recipes:程序员测试的实用方法,Manning Publications,2004 年。

Rainsberger, J. B. JUnit Recipes: Practical Methods for Programmer Testing, Manning Publications, 2004.

拉斯穆森,乔纳森。“将 XP 引入全新项目:经验教训”,IEEE Software,2003 年,http://rasmusson.files.wordpress.com/2008/01/s3021.pdf

Rasmusson, Jonathan. “Introducing XP into Greenfield Projects: Lessons Learned,” IEEE Software, 2003, http://rasmusson.files.wordpress.com/2008/01/s3021.pdf.

罗宾斯、斯蒂芬和蒂姆·贾奇。组织行为学要点,第 9 版,Prentice Hall,2007 年。

Robbins, Stephen and Tim Judge. Essentials of Organizational Behavior, 9th Edition, Prentice Hall, 2007.

施瓦伯,肯。使用 Scrum 进行敏捷项目管理,Microsoft Press,2004 年。

Schwaber, Ken. Agile Project Management with Scrum, Microsoft Press, 2004.

Shore、James 和 Shane Warden。敏捷开发的艺术,O'Reilly Media,2007 年。

Shore, James and Shane Warden. The Art of Agile Development, O’Reilly Media, 2007.

索尼,穆克什。“缺陷预防:降低成本和提高质量”,iSixSigma,http://software.isixsigma.com/library/content/c060719b.asp

Soni, Mukesh. “Defect Prevention: Reducing Costs and Enhancing Quality,” iSixSigma, http://software.isixsigma.com/library/content/c060719b.asp.

萨姆雷尔,梅根。“'Shout-Out' 鞋盒 – 提升团队士气”,http://megansumrell.wordpress.com/2007/08/27/shout-out-shoebox-boosting-team-morale,2007年。

Sumrell, Megan. “’Shout-Out’ Shoebox – Boosting Team Morale,” http://megansumrell.wordpress.com/2007/08/27/shout-out-shoebox-boosting-team-morale, 2007.

Sutherland、Jeff、Carsten Ruseng Jakobsen 和 Kent Johnson。“Scrum 和 CMMI 第 5 级:代码勇士的魔药”,Agile 2007,华盛顿特区,2007 年,http://jeffsutherland.com/scrum/Sutherland-ScrumCMMI6pages.pdf

Sutherland, Jeff, Carsten Ruseng Jakobsen, and Kent Johnson. “Scrum and CMMI Level 5: The Magic Potion for Code Warriors,” Agile 2007, Washington, DC, 2007, http://jeffsutherland.com/scrum/Sutherland-ScrumCMMI6pages.pdf.

塔巴卡,让。协作说明:软件项目负责人的促进技巧,Addison-Wesley,2006 年。

Tabaka, Jean. Collaboration Explained: Facilitation Skills for Software Project Leaders, Addison-Wesley, 2006.

托马斯,迈克。“扼杀遗留代码”,Better Software杂志,2005 年 10 月,http://samoht.com/wiki_downloads/StranglingLegacyCodeArticle.pdf

Thomas, Mike. “Strangling Legacy Code,” Better Software magazine, October 2005, http://samoht.com/wiki_downloads/StranglingLegacyCodeArticle.pdf.

托尔夫森,迈克。“客户拥护者的崛起”,STAREAST,2008 年 5 月 7-9 日。

Tholfsen, Mike. “The Rise of the Customer Champions,” STAREAST, May 7–9, 2008.

沃里斯,约翰。用于外部原型制作和测试的 ADEPT AS400 显示器,www.AdeptTesting.org

Voris, John. ADEPT AS400 Displays for External Prototyping and Testing, www.AdeptTesting.org.

醒醒,比尔。“XP 雷达图”,http://xp123.com/xplor/xp0012b/index.shtml,2001年。

Wake, Bill. “XP Radar Chart,” http://xp123.com/xplor/xp0012b/index.shtml, 2001.

Wilson-Welsh、Patrick 和 Lisa Crispin,“翻转三角形:通向最佳、成本最低的自动化测试的路径”,Agile 2008,2008,http: //patrickwilsonwelsh.com/wp-content/uploads/2008/08/flipping-三角形.pdf

Wilson-Welsh, Patrick and Lisa Crispin, “Flipping the Triangle: Paths to Best, Least-Cost Automated Testing”, Agile 2008, 2008, http://patrickwilsonwelsh.com/wp-content/uploads/2008/08/flipping-the-triangle.pdf.

弗里恩斯,基督。“使用 XP@Scrum 认证 CMM 2 级和 ISO9001”,载于 ADC 2003:敏捷开发会议记录,2003 年 6 月 25-28 日,美国犹他州盐湖城,120-124,IEEE,2003 年。

Vriens, Christ. “Certifying for CMM Level 2 and ISO9001 with XP@Scrum,” in ADC 2003: Proceedings of the Agile Development Conference, 25–28 June 2003, Salt Lake City, UT, USA, 120–124, IEEE, 2003.

工具参考

Tool References

Abbot Java GUI 测试框架,http://abbot.sourceforge.net/doc/overview.shtml

Abbot Java GUI Test Framework, http://abbot.sourceforge.net/doc/overview.shtml.

阿兹克,戈伊科。DbFit:测试驱动的数据库开发,http://gojko.net/fitnesse/dbfit/

Adzik, Gojko. DbFit: Test-driven Database Development, http://gojko.net/fitnesse/dbfit/.

Fauto,丹尼。“测试工具列表”,http ://testingfaqs.org,2008 年。

Faught, Danny. “Test Tools List,” http://testingfaqs.org, 2008.

Canoo WebTest,用于自动测试 Web 应用程序的开源工具,http://webtest.canoo.com

Canoo WebTest, Open Source Tool for Automated Testing of Web Applications, http://webtest.canoo.com.

easyb,Java 平台的行为驱动开发框架,www.easyb.org/

easyb, Behavior Driven Development Framework for the Java Platform, www.easyb.org/.

Fit,集成测试框架,http://fit.c2.com

Fit, Framework for Integrated Test, http://fit.c2.com.

JUnit,测试驱动开发资源,www.junit.org

JUnit, Resources for Test-Driven Development, www.junit.org.

JUnitPerf,用于性能和可扩展性测试的 JUnit 测试装饰器,http://clarkware.com/software/JUnitPerf.html

JUnitPerf, JUnit Test Decorators for Performance and Scalability Testing, http://clarkware.com/software/JUnitPerf.html.

FitNesse,完全集成的独立 Wiki 和验收测试框架,www.fitnesse.org

FitNesse, Fully Integrated Standalone Wiki and Acceptance Testing Framework, www.fitnesse.org.

Hower, Rick,软件 QA 和测试工具信息,www.softwareqatest.com/qattls1.html

Hower, Rick, Software QA and Testing Tools Info, www.softwareqatest.com/qattls1.html.

NUnit,.NET 语言的单元测试框架,http://nunit.org/index.php

NUnit, Unit-testing Framework for .NET Languages, http://nunit.org/index.php.

开源软件测试工具、新闻和讨论。www.opensourcetesting.org/

Open Source Software Testing Tools, News and Discussion. www.opensourcetesting.org/.

RpgUnit,RPG 回归测试框架,www.RPGunit.org

RpgUnit, RPG Regression Testing Framework, www.RPGunit.org.

Selenium,W​​eb 应用程序测试系统,http://seleniumhq.org/

Selenium, Web Application Testing System, http://seleniumhq.org/.

soapUI,Web 服务测试工具,www.soapui.org

soapUI, Web Services Testing Tool, www.soapui.org.

源配置管理,http://better-scm.berlios.de

Source Configuration Management, http://better-scm.berlios.de.

Subversion,开源版本控制系统,http://subversion.tigris.org/

Subversion, Open Source Version Control System, http://subversion.tigris.org/.

单元测试框架。http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks

Unit Testing Frameworks. http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks.

Watir,Ruby 中的 Web 应用程序测试,www.watir.com

Watir, Web Application Testing in Ruby, www.watir.com.

指数

Index

A

A

Abbot GUI 测试工具,127

Abbot GUI test tool, 127

验收测试。另请参阅 面向业务的测试

Acceptance tests. See also Business-facing tests

定义,501

definition, 501

远程数据监控系统示例,245

Remote Data Monitoring system example, 245

UAT(用户验收测试)相比,130

UAT (user acceptance testing) compared with, 130

临时测试,198

Ad hoc testing, 198

适应能力、技能和,39–40

Adaptability, skills and, 39–40

ADEPT(用于外部原型制作和测试的 AS400 显示器),117–118

ADEPT (AS400 Displays for External Prototyping and Testing), 117–118

提前澄清

Advance clarity

客户用同一种声音说话,373–374

customers speaking with one voice, 373–374

确定故事大小,375–376

determining story size, 375–376

收集有关需求的所有观点,374-375

gathering all viewpoints regarding requirements, 374–375

概述, 140–142 , 373

overview of, 140–142, 373

提前准备

Advance preparation

缺点,373

downside of, 373

需要多少,372–373

how much needed, 372–373

敏捷开发

Agile development

敏捷宣言和,3-4

Agile manifesto and, 3–4

的障碍。请参阅 采用敏捷开发的障碍

barriers to. See Barriers to adopting agile development

团队定位,6

team orientation of, 6

敏捷估算和规划(Cohn), 331 , 332

Agile Estimating and Planning (Cohn), 331, 332

敏捷宣言

Agile manifesto

以人为本,30

people focus, 30

声明,4

statement of, 4

中的价值陈述,21

value statements in, 21

敏捷原则。请参阅 适用于敏捷测试人员的原则

Agile principles. See Principles, for agile testers

敏捷测试人员。另见 测试员

Agile testers. See also Testers

敏捷测试心态,482–483

agile testing mind-set, 482–483

定义,4

definition, 4

给予所有团队成员同等的权重,31

giving all team members equal weight, 31

招聘,67–69 岁

hiring, 67–69

他们是什么,19-20

what they are, 19–20

敏捷测试

Agile testing

定义,6

definition, 6

作为心态,20-21

as mind-set, 20–21

我们的意思,4-7

what we mean, 4–7

敏捷价值观,3–4

Agile values, 3–4

Alcea 的 FIT IssueTrack,84

Alcea’s FIT IssueTrack, 84

阿尔法测试,466–467

Alpha tests, 466–467

蚂蚁,284

ant, 284

作为构建工具,126

as build tool, 126

持续构建和175,291

continual builds and, 175, 291

蚁丘临, 126

AnthillPro, 126

ANTS Profiler Pro, 234

ANTS Profiler Pro, 234

阿帕奇 JMeter。参见 JMeter

Apache JMeter. See JMeter

API 层功能测试工具,168–170

API-layer functional test tools, 168–170

Fit 和 FitNesse,168–170

Fit and FitNesse, 168–170

概述, 168

overview of, 168

测试网络服务,170

testing web Services, 170

API测试

API testing

自动化, 282

automating, 282

概述,205-206

overview of, 205–206

API(应用程序编程接口),501

APIs (application programming interfaces), 501

阿普尔顿,布拉德,124 岁

Appleton, Brad, 124

被测应用程序 (AUT),246

Application under test (AUT), 246

应用

Applications

与外部应用程序的集成测试,459

integration testing with external applications, 459

远程数据监控系统示例,242–243

Remote Data Monitoring system example, 242–243

建筑学

Architecture

增量测试方法,114

incremental approach to testing, 114

分层,116

layered, 116

象限 1 测试和,99

Quadrant 1 tests and, 99

可扩展性和104 , 221

scalability and, 104, 221

可测试, 30 , 115 , 182 , 184 , 267

testable, 30, 115, 182, 184, 267

用于外部原型制作和测试的 AS400 显示器 (ADEPT),117–118

AS400 Displays for External Prototyping and Testing (ADEPT), 117–118

假设,隐藏

Assumptions, hidden

敏捷测试人员对25 的回应

agile testers response to, 25

未能检测到,32

failure to detect, 32

发现的问题,136

questions that uncover, 136

最坏情况和334

worst-case scenarios and, 334

态度

Attitude

敏捷测试心态,482–483

agile testing mind-set, 482–483

采用敏捷开发的障碍,48

barriers to adopting agile development, 48

与技能,20

vs. skills, 20

审计,遵守审计要求,89-90

Audits, compliance with audit requirements, 89–90

AUT 被测应用程序143、225、246、317

AUT (application under test), 143, 225, 246, 317

授权、安全测试和,224

Authorization, security testing and, 224

自动化回归测试

Automated regression testing

关键成功因素,484

key success factors, 484

发布候选人和458

release candidates and, 458

作为安全网,261–262

as a safety net, 261–262

自动化测试列表,测试计划备选方案,353–354

Automated test lists, test plan alternatives, 353–354

自动化

Automation

代码通量和,269

code flux and, 269

部署,232

of deployment, 232

推动发展,262–263

driving development with, 262–263

探索性测试,201

of exploratory testing, 201

害怕,269–270

fear of, 269–270

来自262的反馈

feedback from, 262

解放人们从事其他工作,259–261

freeing people for other work, 259–261

功能测试结构,245–247

of functional test structure, 245–247

自制测试,175

home-brewed test, 175

所需投资,267–268

investment required, 267–268

学习曲线,266–267

learning curve, 266–267

遗留代码和,269

legacy code and, 269

可维护性和227–228

maintainability and, 227–228

手动测试与258–259

manual testing vs., 258–259

障碍,264–265

obstacles to, 264–265

旧习惯,270

old habits and, 270

概述, 255

overview of, 255

程序员的态度,265–266

programmers’ attitude regarding, 265–266

原因,257-258

reasons for, 257–258

应对变化,29

responding to change and, 29

投资回报率和264

ROI and, 264

任务卡和394–395

task cards and, 394–395

可测试性149–150

testability and, 149–150

测试作为文档,263–264

tests as documentation, 263–264

自动化策略

Automation strategy

敏捷编码实践和303–304

agile coding practices and, 303–304

一次使用一个工具,312–313

applying one tool at a time, 312–313

数据生成工具,304–305

data generation tools, 304–305

数据库访问和,306-310

database access and, 306–310

设计和维护以及292–294

design and maintenance and, 292–294

发展中,288–289

developing, 288–289

确定工具要求,311–312

identifying tool requirements, 311–312

实施,316–319

implementing, 316–319

迭代方法,299–300

iterative approach, 299–300

保持简单,298–299

keep it simple, 298–299

边做边学,303

learning by doing, 303

管理自动化测试,319

managing automated tests, 319

多层方法,290-292

multi-layered approach to, 290–292

组织测试结果,322–324

organizing test results, 322–324

组织测试,319–322

organizing tests, 319–322

概述, 273

overview of, 273

原则,298

principles, 298

记录/回放工具和, 294 , 296–297

record/playback tools and, 294, 296–297

从最痛的区域开始,289-290

starting with area of greatest pain, 289–290

花时间做对,301-303

taking time to do it right, 301–303

测试自动化金字塔,276–279

test automation pyramid, 276–279

测试类别,274–276

test categories, 274–276

工具选择294-298,313-316

tool selection, 294–298, 313–316

理解测试的目的,310–311

understanding purpose of tests and, 310–311

什么可以自动化,279–285

what can be automated, 279–285

什么可能难以自动化,287–288

what might be difficult to automate, 287–288

什么不应该自动化,285–287

what should not be automated, 285–287

整个团队方法,300–301

whole team approach, 300–301

自动化工具,164–177

Automation tools, 164–177

API 层功能测试工具,168–170

API-layer functional test tools, 168–170

建立和,126

builds and, 126

GUI 测试工具,170–176

GUI test tools, 170–176

概述,164-165

overview of, 164–165

单元级测试工具,165–168

unit-level test tools, 165–168

Web 服务测试工具,170

web services test tool, 170

B

巴赫,詹姆斯,195 , 200 , 212

Bach, James, 195, 200, 212

巴赫,乔纳森,201

Bach, Jonathan, 201

后端测试

Back-end testing

在 GUI 后面,282

behind the GUI, 282

非 UI 测试,204–205

non-UI testing, 204–205

竹子,126

Bamboo, 126

采用敏捷开发的障碍,44–49

Barriers to adopting agile development, 44–49

冲突或多重角色,45

conflicting or multiple roles, 45

角色之间的文化差异,48–49

cultural differences among roles, 48–49

缺乏培训,45

lack of training, 45

缺乏对敏捷概念的理解,45–48

lack of understanding of agile concepts, 45–48

失去身份,44-45

loss of identity, 44–45

概述, 44

overview of, 44

过去的经验和态度,48

past experience and attitudes, 48

基线

Baselines

断裂测试基线技术,363

break-test baseline technique, 363

性能,235–237

performance, 235–237

Batch

档案, 251

files, 251

处理, 345

processing, 345

调度过程,182

scheduling process, 182

BDD(行为驱动开发)

BDD (Behavior-driven development)

easyb 工具,166–168

easyb tool, 166–168

象限 1 测试工具,127

tools for Quadrant 1 tests, 127

贝克,肯特,26 岁99 岁

Beck, Kent, 26, 99

贝南德,马克,51 岁

Benander, Mark, 51

基准测试,237

Benchmarking, 237

Berczuk, 斯蒂芬, 124

Berczuk, Stephen, 124

Beta 测试,466–467

Beta testing, 466–467

大图

Big picture

敏捷测试人员专注于,23

agile testers focus on, 23

高级测试和示例,397–402

high-level tests and examples, 397–402

关键成功因素,490–491

key success factors, 490–491

遗忘的危险,148

peril of forgetting, 148

回归测试和,434

regression tests and, 434

迈克尔·博尔顿,195 岁

Bolton, Michael, 195

博斯,埃里克,114

Bos, Erik, 114

边界条件

Boundary conditions

API测试和,205

API testing and, 205

自动化和, 11

automation and, 11

数据生成工具和,304

data generation tools and, 304

识别测试变体,410

identifying test variations, 410

为137编写测试用例

writing test cases for, 137

博耶埃里卡140、163、372、432 _ _ _

Boyer, Erika, 140, 163, 372, 432

头脑风暴

Brainstorming

自动化让测试人员更好地工作,260

automation giving testers better work, 260

在迭代之前,370、381

prior to iteration, 370, 381

象限作为框架,253

quadrants as framework for, 253

花时间,301

taking time for, 301

测试人员,121

testers, 121

断裂测试基线技术,363

Break-test baseline technique, 363

浏览器、兼容性测试和230

Browsers, compatibility testing and, 230

预算限制,55

Budget limits, 55

错误跟踪。请参见 缺陷跟踪

Bug tracking. See Defect tracking

错误。查看 缺陷

Bugs. See Defects

建造

Build

自动化,280–282

automating, 280–282

具有挑战性的候选发布版本,473

challenging release candidate builds, 473

定义,501

definition, 501

增量,178–179

incremental, 178–179

加速,118–119

speeding up, 118–119

构建自动化工具, 126 , 282

Build automation tools, 126, 282

构建/操作/检查模式,180

Build/Operate/Check pattern, 180

构建工具,126

Build tools, 126

构建节拍,126

BuildBeat, 126

业务分析师,374

Business analysts, 374

业务专家角色

Business expert role

关于要求的协议, 428 , 430

agreement regarding requirements, 428, 430

通用语言和, 134 , 291 , 414

common language and, 134, 291, 414

在客户团队中,6-7

on customer team, 6–7

迭代演示和,443

iteration demo and, 443

291的语言

language of, 291

三的幂和,482

Power of Three and, 482

适合的工具,134

tools geared to, 134

面向业务的测试

Business-facing tests

敏捷测试,6

agile testing as, 6

象限 2 & 3, 97–98

Quadrants 2 & 3, 97–98

与120相比,面向技术的测试

technology-facing tests compared with, 120

面向业务的测试,批评产品(第 3 象限),189–215

Business-facing tests, critiquing the product (Quadrant 3), 189–215

验收测试,245

acceptance tests, 245

API 测试,205–206

API testing, 205–206

示威,191–192

demonstrations, 191–192

仿真器工具,213–214

emulator tools, 213–214

端到端测试,249–250

end-to-end tests, 249–250

探索性测试, 195–202 , 248–249

exploratory testing, 195–202, 248–249

生成测试数据,212

generating test data, 212

GUI 测试,204

GUI testing, 204

监控工具,212-213

monitoring tools, 212–213

概述,189-191

overview of, 189–191

报告,208–210

reports, 208–210

场景测试,192–195

scenario testing, 192–195

基于会话的测试,200-201

session-based testing, 200–201

设置测试,211-212

setting up tests, 211–212

模拟器工具,213

simulator tools, 213

探索性测试工具,210–211

tools for exploratory testing, 210–211

可用性测试,202–204

usability testing, 202–204

用户验收测试,250

user acceptance testing, 250

用户文档,207-208

user documentation, 207–208

Web 服务测试,207

web services testing, 207

面向业务的测试,支持团队(象限 2),129–151

Business-facing tests, supporting team (Quadrant 2), 129–151

提高清晰度,140–142

advance clarity, 140–142

自动化功能测试,245–247

automating functional tests, 245–247

共同语言和,134-135

common language and, 134–135

满意的条件,142-143

conditions of satisfaction and, 142–143

熟度,146–147

doneness, 146–147

推动发展,129–132

driving development with, 129–132

引出要求,135–140

eliciting requirements, 135–140

嵌入式测试,248

embedded testing, 248

渐进方法,144-146

incremental approach, 144–146

需求窘境,132-134

requirements quandary and, 132–134

涟漪效应,143–144

ripple effects, 143–144

风险缓解和147–149

risk mitigation and, 147–149

可测试性和自动化,149–150

testability and automation, 149–150

的工具包。 工具包(象限 2)

toolkit for. See Toolkit (Quadrant 2)

Web 服务测试,247–248

web services testing, 247–248

业务影响,475–476

Business impact, 475–476

商业价值

Business value

增加价值,31–33

adding value, 31–33

作为敏捷开发的目标,5–8 , 69 , 454

as goal of agile development, 5–8, 69, 454

指标和,75

metrics and, 75

发布周期和3

release cycles and, 3

角色、功能、商业价值模式, 155

role, function, business value pattern, 155

团队方法和16

team approach and, 16

布塞迈克106、235、284、313 _ _ _

Busse, Mike, 106, 235, 284, 313

布瓦尔达,汉斯,193 岁

Buwalda, Hans, 193

C

C

规范数据、自动化数据库和308–309

Canonical data, automating databases and, 308–309

Canoo Web测试

Canoo WebTest

自动化 GUI测试184,186

automating GUI tests, 184, 186

GUI 回归测试套件,291

GUI regression test suite, 291

GUI 冒烟测试,300

GUI smoke tests, 300

GUI 测试工具,174–175

GUI test tools, 174–175

组织测试和,320

organizing tests and, 320

脚本和320

scripts and, 320

XML 编辑器,125

XML Editor for, 125

能力成熟度模型集成 (CMMI),90–91

Capability Maturity Model Integration (CMMI), 90–91

抓拍回放工具,267

Capture-playback tool, 267

庆祝成功

Celebrating successes

改变实施,50-52

change implementation and, 50–52

迭代总结和,449-451

iteration wrap up and, 449–451

钱德拉,阿普瓦,377

Chandra, Apurva, 377

张泰,53–54 岁

Chang, Tae, 53–54

改变

Change

庆祝成功,50-52

celebrating successes, 50–52

赋予团队所有权,50

giving team ownership, 50

介绍, 49

introducing, 49

来之不易,56–57

not coming easy, 56–57

28–29 的反应

responsiveness to, 28–29

谈论恐惧,49–50

talking about fears, 49–50

清单

Checklists

发布准备,474

release readiness, 474

引出示例和要求的工具,156

tools for eliciting examples and requirements, 156

CI。请参阅 持续集成(CI)

CI. See Continuous integration (CI)

CI厂,126

CI Factory, 126

CMMI(能力成熟度模型集成),90–91

CMMI (Capability Maturity Model Integration), 90–91

同地办公、团队后勤和65–66

Co-location, team logistics and, 65–66

教练员

Coaches

适应敏捷文化,40

adjusting to agile culture and, 40

学习曲线和,266

learning curve and, 266

提供鼓励,69

providing encouragement, 69

技能发展和122

skill development and, 122

培训和,45-46

training and, 45–46

科伯恩,阿利斯泰尔,115

Cockburn, Alistair, 115

代码

Code

自动化和代码通量,269

automation and code flux, 269

自动化和遗留代码,269

automation and legacy code, 269

自动化策略和,303-304

automation strategy and, 303–304

251的文档

documentation of, 251

标准,227

standards, 227

写作可测试,115

writing testable, 115

代码覆盖率、发布指标,360–364

Code coverage, release metrics, 360–364

编码和测试,405–441

Coding and testing, 405–441

增加复杂性,407

adding complexity, 407

处理错误的替代方法,424–428

alternatives for dealing with bugs, 424–428

选择何时修复错误,421–423

choosing when to fix bugs, 421–423

与程序员合作,413–414

collaborating with programmers, 413–414

处理错误,416–419

dealing with bugs, 416–419

决定记录哪些错误,420–421

deciding which bugs to log, 420–421

推动发展,406

driving development and, 406

促进沟通,429-432

facilitating communication, 429–432

专注于一个故事,411-412

focusing on one story, 411–412

识别变异,410

identifying variations, 410

迭代指标,435–440

iteration metrics, 435–440

用于记录错误的媒体,423–424

media for logging bugs, 423–424

概述, 405

overview of, 405

解决观点分歧的三的力量,411

Power of Three for resolving differences in viewpoint, 411

回归测试和432–434

regression testing and, 432–434

资源,434–435

resources, 434–435

风险评估,407-409

risk assessment, 407–409

作为同步过程409-410,488-489

as simultaneous process, 409–410, 488–489

从简单开始406、428–429

starting simple, 406, 428–429

与客户交谈,414–415

talking to customers, 414–415

批评产品的测试,412–413

tests that critique the product, 412–413

科恩、迈克,50 岁155 岁276 岁296岁、331 岁332 岁

Cohn, Mike, 50, 155, 276, 296, 331, 332

合作

Collaboration

与客户,396-397

with customers, 396–397

关键成功因素,489–490

key success factors, 489–490

与程序员,413-414

with programmers, 413–414

整个团队的方法,15-16

whole team approach, 15–16

科里诺,亚历山德罗103,363

Collino, Alessandro, 103, 363

沟通

Communication

共同语言和,134-135

common language and, 134–135

与客户, 140 , 396–397

with customer, 140, 396–397

DTS(缺陷跟踪系统)和,83

DTS (Defect Tracking System) and, 83

促进, 23–25 , 429–432

facilitating, 23–25, 429–432

产品交付和462–463

product delivery and, 462–463

尺寸作为挑战,42-43

size as challenge to, 42–43

团队之间,69–70

between teams, 69–70

测试结果,357–358

test results, 357–358

比较,自动化,283

Comparisons, automating, 283

兼容性测试,229–230

Compatibility testing, 229–230

组件测试

Component tests

自动化, 282

automating, 282

定义,501

definition, 501

支持功能,5

supporting function of, 5

满意条件

Conditions of satisfaction

面向业务的测试,以及142–143

business-facing tests and, 142–143

定义,501–502

definition, 501–502

上下文驱动测试

Context-driven testing

定义,502

definition, 502

象限和106–107

quadrants and, 106–107

持续构建过程

Continuous build process

失败通知和112

failure notification and, 112

反馈和,119

feedback and, 119

FitNesse 测试和357

FitNesse tests and, 357

实施,114

implementing, 114

将工具与175311集成

integrating tools with, 175, 311

源代码控制和124

source code control and, 124

测试人员可以做什么,121

what testers can do, 121

持续反馈原则,22

Continuous feedback principle, 22

持续改进原则,27-28

Continuous improvement principle, 27–28

持续集成 (CI)

Continuous integration (CI)

自动化,280–282

automating, 280–282

作为核心实践,486–487

as core practice, 486–487

可安装性和231–232

installability and, 231–232

远程数据监控系统示例,244

Remote Data Monitoring system example, 244

运行测试和,111–112

running tests and, 111–112

转换、数据迁移和460–461

Conversion, data migration and, 460–461

核心实践

Core practices

编码和测试作为一个过程,488–489

coding and testing as one process, 488–489

持续集成,486–487

continuous integration, 486–487

渐进方法,488

incremental approach, 488

概述, 486

overview of, 486

实践之间的协同作用,489

synergy between practices, 489

技术债务管理,487–488

technical debt management, 487–488

测试环境,487

test environments, 487

勇气,原则25-26,71

Courage, principles, 25–26, 71

信誉建设,57

Credibility, building, 57

批评产品

Critiquing the product

企业面临考验。请参阅 面向业务的测试,评价产品(第 3 象限)

business facing tests. See Business-facing tests, critiquing the product (Quadrant 3)

面向技术的测试。请参阅 面向技术的测试,批评产品(象限 4)

technology-facing tests. See Technology-facing tests, critiquing the product (Quadrant 4)

CrossCheck,测试 Web 服务,170

CrossCheck, testing Web Services, 170

巡航控制, 126 , 244 , 291

CruiseControl, 126, 244, 291

文化变革,37另见 组织

Cultural change, 37. See also Organizations

坎宁安, 沃德, 106 , 168 , 506

Cunningham, Ward, 106, 168, 506

客户期望

Customer expectations

业务影响和475–476

business impact and, 475–476

生产支持,475

production support, 475

面向客户的测试。查看 面向业务的测试

Customer-facing test. See Business-facing tests

客户支持、DTS(缺陷跟踪系统)和82

Customer support, DTS (Defect Tracking System) and, 82

客户团队

Customer team

定义,502

definition, 502

客户和开发团队之间的互动,8

interaction between customer and developer teams, 8

概述,7

overview of, 7

客户测试

Customer testing

Alpha/Beta 测试,466–467

Alpha/Beta testing, 466–467

定义,502

definition, 502

概述, 464

overview of, 464

UAT(用户验收测试),464–466

UAT (user acceptance testing), 464–466

顾客

Customers

合作, 396–397 , 489–490

collaborating with, 396–397, 489–490

在迭代规划期间考虑所有观点,388–389

considering all viewpoints during iteration planning, 388–389

22-23 岁的人提供价值

delivering value to, 22–23

140 , 414–415 , 444沟通的重要性

importance of communicating with, 140, 414–415, 444

迭代演示,191–192,443–444

iteration demo, 191–192, 443–444

参与迭代计划,384–385

participation in iteration planning,384–385

关系, 41–42

relationship with, 41–42

审查高级测试,400

reviewing high-level tests with, 400

用同一个声音说话,373–374

speaking with one voice, 373–374

CVS、源代码控制和124

CVS, source code control and, 124

D

数据

Data

自动创建或设置,284–285

automating creation or setup, 284–285

清理, 461

cleanup, 461

转换,459–461

conversion, 459–461

发布计划和348

release planning and, 348

写任务卡和,392

writing task cards and, 392

数据驱动测试,182–183

Data-driven tests, 182–183

数据馈送、测试、249

Data feeds, testing, 249

数据生成工具,304-305

Data generation tools, 304–305

数据迁移、自动化310、460

Data migration, automating, 310, 460

数据库

Databases

运行测试时避免访问,306–310

avoiding access when running tests, 306–310

规范数据和自动化,308–309

canonical data and automation, 308–309

可维护性和,228

maintainability and, 228

产品交付和更新,459–461

product delivery and updates, 459–461

类似生产的数据和自动化,309–310

production-like data and automation, 309–310

为每个自动化测试设置/拆除数据,307–308

setting up/tearing down data for each automated test, 307–308

测试数据迁移,310

testing data migration, 310

德索萨,肯,223

De Souza, Ken, 223

截止日期、范围和340–341

Deadlines, scope and, 340–341

缺陷指标

Defect metrics

概述,437–440

overview of, 437–440

发布指标,364–366

release metrics, 364–366

缺陷跟踪,79-86

Defect tracking, 79–86

DTS(缺陷跟踪系统),79-83

DTS (Defect Tracking System), 79–83

保持专注,85–86

keeping focus and, 85–86

概述, 79

overview of, 79

原因,79

reasons for, 79

83–85的工具

tools for, 83–85

缺陷跟踪系统。参见 DTS(缺陷跟踪系统)

Defect Tracking System. See DTS (Defect Tracking System)

缺陷

Defects

处理错误的替代方法,424–428

alternatives for dealing with bugs,424–428

选择何时修复错误,421–423

choosing when to fix bugs, 421–423

处理错误,416–419

dealing with bugs, 416–419

决定记录哪些错误,420–421

deciding which bugs to log, 420–421

用于记录错误的媒体,423–424

media for logging bugs, 423–424

指标和,79

metrics and, 79

TDD(测试驱动开发)和,490

TDD (test-driven development) and, 490

写任务卡和,391-392

writing task cards and, 391–392

零错误容忍, 79 , 418–419

zero bug tolerance, 79, 418–419

可交付成果

Deliverables

“安装并完成”可交付成果,454

“fit and finish” deliverables, 454

非软件,470

nonsoftware, 470

概述,468–470

overview of, 468–470

交付产品

Delivering product

Alpha/Beta 测试,466–467

Alpha/Beta testing, 466–467

业务影响和475–476

business impact and, 475–476

通讯和,462-463

communication and, 462–463

客户期望,475

customer expectations, 475

客户测试,464

customer testing, 464

数据转换和数据库更新,459–461

data conversion and database updates, 459–461

可交付成果,468–470

deliverables, 468–470

游戏结束,456–457

end game, 456–457

安装测试,461–462

installation testing, 461–462

与外部应用程序集成,459

integration with external applications, 459

非功能性测试和458–459

nonfunctional testing and, 458–459

概述, 453

overview of, 453

包装,474–475

packaging, 474–475

计划测试时间,455–456

planning time for testing, 455–456

开发后测试周期,467–468

post-development testing cycles, 467–468

生产支持,475

production support, 475

发布验收标准,470–473

release acceptance criteria, 470–473

发布管理, 470 , 474

release management, 470, 474

发布产品,470

releasing product, 470

暂存环境和,458

staging environment and, 458

测试候选版本,458

testing release candidates, 458

UAT(用户验收测试),464–466

UAT (user acceptance testing), 464–466

如果还没有准备好怎么办,463–464

what if it is not ready, 463–464

是什么造就了产品,453–455

what makes a product, 453–455

演示/示范

Demos/demonstrations

迭代次数,443–444

of an iteration, 443–444

对客户的价值,191–192

value to customers, 191–192

部署、自动化,280–282

Deployment, automating, 280–282

设计

Design

自动化策略和,292–294

automation strategy and, 292–294

设计时考虑到测试,115–118

designing with testing in mind, 115–118

详细的测试用例

Detailed test cases

写作艺术与科学,178

art and science of writing, 178

大局观和,148-149

big picture approach and, 148–149

设计,401

designing with, 401

开发团队

Developer team

客户和开发团队之间的互动,8

interaction between customer and developer teams, 8

概述,7-8

overview of, 7–8

发展

Development

敏捷开发, 3–4 , 6

agile development, 3–4, 6

自动驾驶测试,262–263

automated tests driving, 262–263

面向业务的驾驶测试,129–132

business-facing tests driving, 129–132

编码驾驶,406

coding driving, 406

开发后测试周期,467–468

post-development testing cycles, 467–468

发展高峰,381

Development spikes, 381

开发团队,502

Development team, 502

差异工具,283

diff tool, 283

分布式团队,431–432

Distributed teams, 431–432

缺陷追踪系统,82

defect tracing systems, and, 82

实体物流,66

physical logistics, 66

在线高级测试,399

online high level tests for, 399

在线故事板,357

online story board for, 357

应对变化,29

responding to change, 29

基于软件的工具来引出示例和需求,以及163–164

software-based tools to elicit examples and requirements, and, 163–164

文档

Documentation

自动化测试作为来源,263–264

automated tests as source of, 263–264

问题和修复,417

problems and fixes, 417

报告,208–210

reports, 208–210

测试代码,251

of test code, 251

测试为,402

tests as, 402

用户文档,207-208

user documentation, 207–208

熟度

Doneness

知道故事何时结束,104–105

knowing when a story is done, 104–105

多层, 471–472

multitiered, 471–472

通过测试推动开发。请参见TDD(测试驱动开发)

Driving development with tests. See TDD (test-driven development)

DTS(缺陷跟踪系统),80-83

DTS (Defect Tracking System), 80–83

80-82的好处

benefits of, 80–82

选择用于记录错误的媒体,424

choosing media for logging bugs, 424

记录问题和修复,417

documenting problems and fixes, 417

记录错误和420

logging bugs and, 420

不使用的原因,82-83

reason for not using, 82–83

戴蒙德、罗宾、xxx

Dymond, Robin, xxx

动态分析,安全测试工具,225

Dynamic analysis, security testing tools, 225

E

easyb 行为驱动开发工具,165–168

easyb behavior-driven development tool, 165–168

轻松模拟,127

EasyMock, 127

日食, 125 , 316

Eclipse, 125, 316

边缘案例

Edge cases

识别变异,410

identifying variations, 410

没时间,112

not having time for, 112

从简单开始,然后增加复杂性,406–407

starting simple and then adding complexity, 406–407

测试用例,137

test cases for, 137

嵌入式系统,远程数据监控实例,248

Embedded system, Remote Data Monitoring example, 248

赋权,团队,44

Empowerment, of teams, 44

仿真器工具,213–214

Emulator tools, 213–214

结束游戏

End game

敏捷测试,91

Agile testing, 91

迭代,14

iteration, 14

产品交付和456–457

product delivery and, 456–457

释放和,327

release and, 327

端到端测试,249–250

End-to-end tests, 249–250

享受,原则,31

Enjoyment, principle of, 31

环境,测试环境,347-348

Environment, test environment, 347–348

史诗。另见 主题

Epic. See also Themes

定义,502

definition, 502

功能成为,502

features becoming, 502

迭代76,329 _

iterations in, 76, 329

规划, 252

planning, 252

ePlan Services, Inc., xli, 267

ePlan Services, Inc., xli, 267

错误,手动测试和,259

Errors, manual testing and, 259

估计故事大小,332–338

Estimating story size, 332–338

有效, 234

eValid, 234

基于事件的模式,测试设计模式,181

Event-based patterns, test design patterns, 181

为团队、测试人员和您编写 Ruby 日常脚本(Marick), 297 , 303

Everyday Scripting with Ruby for Teams, Testers, and You (Marick), 297, 303

示例驱动开发,378–380

Example-driven development, 378–380

例子

Examples

用于引发需求,136-137

for eliciting requirements, 136–137

引出示例和要求的工具,155-156

tools for eliciting examples and requirements, 155–156

可执行测试,406

Executable tests, 406

探索性测试(ET)

Exploratory testing (ET)

活动、特征和技能(夏甲),198–200

activities, characteristics, and skills (Hagar), 198–200

探索性测试者的属性,201–202

attributes of exploratory tester, 201–202

自动化,201

automation of, 201

定义,502–503

definition, 502–503

结束游戏,457

end game and, 457

解释(博尔顿),195-198

explained (Bolton), 195–198

手动测试和,280

manual testing and, 280

监控工具, 212

monitoring tools, 212

概述, 26 , 195

overview of, 26, 195

远程数据监控系统示例,248–249

Remote Data Monitoring system example, 248–249

基于会话的测试和200–201

session-based testing and, 200–201

设置,211–212

setup, 211–212

模拟器和模拟器,212-213

simulators and emulators, 212–213

批评产品的测试,412–413

tests that critique the product, 412–413

工具,210–212

tools for, 210–212

用于生成测试数据的工具,212

tools for generating test data, 212

什么不应该自动化,286

what should not be automated, 286

外部质量,面向业务的测试定义99,131

External quality, business facing tests defining, 99, 131

外部团队, 43 , 457

External teams, 43, 457

极限编程。参见 XP(极限编程)

Extreme Programming. See XP (Extreme Programming)

极限编程解释(Beck),26

Extreme Programming Explained (Beck), 26

F

F

面对面交流,23-25

Face-to-face communication, 23–25

故障转移测试,232

Failover tests, 232

失败,勇于学习,25

Failure, courage to learn from, 25

假对象,115 , 118 , 306 , 502–503

Fake objects, 115, 118, 306, 502–503

容错、产品交付和、459

Fault tolerance, product delivery and, 459

害怕

Fear

自动化的障碍,269–270

barriers to automation, 269–270

改变,49-50

change and, 49–50

无所畏惧的变化(Manns 和 Rising),121

Fearless Change (Manns and Rising), 121

羽毛,迈克尔117,288

Feathers, Michael, 117, 288

特征

Features

缺陷对比,417–418

defects vs., 417–418

定义,502–503

definition, 502–503

关注价值,341

focusing on value, 341

反馈

Feedback

自动化测试提供,262

automated tests providing, 262

持续反馈原则,22

continuous feedback principle, 22

迭代方法和,299-300

iterative approach and, 299–300

关键成功因素,484–486

key success factors, 484–486

管理测试,323–324

managing tests for, 323–324

象限 1 测试和,118–119

Quadrant 1 tests and, 118–119

“安装并完成”可交付成果,454

“Fit and finish” deliverables, 454

Fit(集成测试框架),134–135

Fit (Framework for Integrated Test), 134–135

API 层功能测试工具,168–169

API-layer functional test tools, 168–169

自动化测试金字塔和,278

automation test pyramid and, 278

FIT IssueTrack,Alcea,83–84

FIT IssueTrack, Alcea, 83–84

健身

FitNesse

163的优势

advantages of, 163

API 层功能测试工具,169–170

API-layer functional test tools, 169–170

自动化功能测试30,145

automating functional tests with, 30, 145

面向业务的测试,154 , 178

business-facing tests with, 154, 178

协作,164

collaboration and, 164

持续构建和119,357

continual builds and, 119, 357

数据验证,287

data verification with, 287

熟度和,472

doneness and, 472

鼓励使用,122

encouraging use of, 122

例子136,169

examples and, 136, 169

反馈和323–324

feedback and, 323–324

文件解析规则说明,205

file parsing rules illustrated with, 205

GUI 背后的功能测试, 291 , 300

functional testing behind the GUI, 291, 300

本土脚本和305

home-grown scripts and, 305

与 JUnit 相比,299

JUnit compared with, 299

用于自动化测试的关键字或操作词,182–183

keywords or actions words for automating tests, 182–183

手动与自动测试,210

manual vs. automated testing, 210

内存需求,306

memory demands of, 306

组织测试和319–320

organizing tests and, 319–320

概述,168–170

overview of, 168–170

远程测试和,432

remote testing and, 432

“开始、停止、继续”列表,446

“start, stop, continue” list, 446

支持源代码控制工具,320

support for source code control tools, 320

测试自动化金字塔和,278

test automation pyramid and, 278

测试卡和,389–390

test cards and, 389–390

测试用例作为文档,402

test cases as documentation, 402

测试设计和维护,292

test design and maintenance, 292

测试数据库层,284

testing database layer with, 284

测试故事,395

testing stories, 395

可追溯性要求和88

traceability requirements and, 88

用户验收测试,295

user acceptance testing, 295

维基和186

wikis and, 186

弗莱施,帕特里克,377,440

Fleisch, Patrick, 377, 440

流程图

Flow diagrams

场景测试和194–195

scenario testing and, 194–195

引出示例和要求的工具,160-163

tools for eliciting examples and requirements, 160–163

马丁·福勒,117

Fowler, Martin, 117

集成测试框架。See Fit(集成测试框架)

Framework for Integrated Test. See Fit (Framework for Integrated Test)

框架,90-93

Frameworks, 90–93

ftptt, 234

ftptt, 234

职能分析师,386

Functional analysts, 386

功能测试

Functional testing

兼容性问题和230

compatibility issues and, 230

定义,502–503

definition, 502–503

端到端测试,249–250

end-to-end tests, 249–250

层数,246

layers, 246

与225相比,非功能测试

nonfunctional tests compared with, 225

远程数据监控系统示例,245–247

Remote Data Monitoring system example, 245–247

G

G

鲍勃·盖伦,455–456 , 471

Galen, Bob, 455–456, 471

Gärtner, Markus, 395 , 476

Gärtner, Markus, 395, 476

地域分散的团队

Geographically dispersed teams

应对,376–378

coping with, 376–378

促进沟通,431-432

facilitating communication and, 431–432

Gheorghiu, 格里格, 225–226 , 234

Gheorghiu, Grig, 225–226, 234

安德鲁·格洛弗,166 岁

Glover, Andrew, 166

绿地项目

Greenfield projects

代码测试和,116

code testing and, 116

定义,502–503

definition, 502–503

GUI(图形用户界面)

GUI (graphical user interface)

自动化策略和,293

automation strategy and, 293

代码通量和,269

code flux and, 269

标准,227

standards, 227

GUI 冒烟测试

GUI smoke tests

Canoo WebTest 和300

Canoo WebTest and, 300

持续构建,119

continual builds and, 119

缺陷指标,437

defect metrics, 437

GUI 测试工具,170–176

GUI test tools, 170–176

Canoo 网络测试,174–175

Canoo Web Test, 174–175

“自制”测试自动化工具,175

“home-brewed” test automation tools, 175

开源测试工具,172

open source test tools, 172

概述,170-171

overview of, 170–171

记录/回放工具,171–172

record/playback tools, 171–172

红宝石与 Watir,172–174

Ruby with Watir, 172–174

硒,174

Selenium, 174

界面测试

GUI testing

API 测试,205–206

API testing, 205–206

自动化, 282–283 , 295–296

automating, 282–283, 295–296

自动化测试金字塔和,278

automation test pyramid and, 278

GUI冒烟测试,119、300、437 _

GUI smoke tests, 119, 300, 437

概述, 204

overview of, 204

Web 服务测试,207

Web service testing, 207

H

H

乔恩·夏甲,198 岁

Hagar, Jon, 198

硬件

Hardware

兼容性和229

compatibility and, 229

测试环境的成本,487

cost of test environments, 487

功能测试和,230

functional testing and, 230

投资自动化,267

investing in automation and, 267

生产环境和,310

production environment and, 310

可扩展性和,233

scalability and, 233

测试基础设施,319

test infrastructure, 319

测试产品安装,462

testing product installation, 462

亨德里克森,伊丽莎白203,315–316

Hendrickson, Elisabeth, 203, 315–316

高级测试用例,397–402

High-level test cases, 397–402

模型,398–399

mockups, 398–399

概述,397–398

overview of, 397–398

与客户一起审查,400

reviewing with customers, 400

与程序员一起审查,400–401

reviewing with programmers, 400–401

测试用例作为文档,402

test cases as documentation, 402

雇用一名测试员,67–69 岁

Hiring a tester, 67–69

霍尔泽,杰森220,448

Holzer, Jason, 220, 448

自主研发的测试工具

Home-grown test tool

自动化工具, 314

automation tools, 314

GUI 测试工具,175

GUI test tools, 175

测试结果,323

test results, 323

HTTPerf,234

httperf, 234

哈德森,126 岁

Hudson, 126

I

IBM Rational ClearCase,124

IBM Rational ClearCase, 124

IDE(集成开发环境)

IDEs (Integrated Development Environments)

定义,502–503

definition, 502–503

日志分析工具,212

log analysis tools, 212

象限 1 测试的工具,124–126

tools for Quadrant 1 tests, 124–126

“能力”测试

“ility” testing

兼容性测试,229–230

compatibility testing, 229–230

安装性测试,231-232

installability testing, 231–232

互操作性测试,228–229

interoperability testing, 228–229

可维护性测试,227-228

maintainability testing, 227–228

可靠性测试, 230–231 , 250–251

reliability testing, 230–231, 250–251

安全测试,223-227

security testing, 223–227

影响,全系统,342

Impact, system-wide, 342

实施精益软件开发:从概念到现金( Poppendieck),74,416

Implementing Lean Software Development: From Concept to Cash (Poppendieck), 74, 416

改进

Improvement

流程改进方法,448–449

approach to process improvement, 448–449

持续改进原则,27-28

continuous improvement principle, 27–28

从回顾中改进的想法,447-449

ideas for improvement from retrospectives, 447–449

增量开发

Incremental development

逐步构建测试,178–179

building tests incrementally, 178–179

作为核心实践,488

as core practice, 488

“能力”测试和232

“ilities” tests and, 232

薄片,小块,144-146

thin slices, small chunks, 144–146

传统与敏捷测试,12–13

traditional vs. agile testing, 12–13

索引卡,记录错误,423

Index cards, logging bugs on, 423

基础设施

Infrastructure

象限 1 测试,111–112

Quadrant 1 tests, 111–112

测试基础设施,319

test infrastructure, 319

测试计划和346–347

test plans and, 346–347

安装性测试,231-232

Installability testing, 231–232

安装测试,461–462

Installation testing, 461–462

集成开发环境。请参见 IDE集成开发环境

Integrated Development Environments. See IDEs (Integrated Development Environments)

集成测试

Integration testing

互操作性和229

interoperability and, 229

产品和外部应用程序,459

product and external applications, 459

IntelliJ IDEA,125

IntelliJ IDEA, 125

内在品质

Internal quality

衡量代码的内部质量,99

measuring internal quality of code, 99

满足团队标准,366

meeting team standards, 366

象限 1 测试和,111

Quadrant 1 tests and, 111

速度和,112

speed and, 112

互操作性测试,228–229

Interoperability testing, 228–229

投资、自动化要求,267–268

Investment, automation requiring, 267–268

迭代

Iteration

自动化策略和,299–300

automation strategy and, 299–300

定义,502–503

definition, 502–503

演示,443–444

demo, 443–444

测试仪的寿命和327

life of a tester and, 327

预迭代活动。请参阅 预迭代活动

pre-iteration activities. See Pre-iteration activities

优先报道,338

prioritizing stories and, 338

评论, 415 , 435–437

review, 415, 435–437

传统与敏捷测试,12–13

traditional vs. agile testing, 12–13

迭代开始,383–403

Iteration kickoff, 383–403

与客户合作,396–397

collaboration with customers, 396–397

考虑所有观点,385-389

considering all viewpoints, 385–389

控制工作量,393

controlling workload, 393

高级测试和示例,397–402

high-level tests and examples, 397–402

迭代计划,383–384

iteration planning, 383–384

学习项目细节,384–385

learning project details, 384–385

概述, 383

overview of, 383

可测试的故事,393–396

testable stories, 393–396

写任务卡,389–392

writing task cards, 389–392

迭代指标,435–440

Iteration metrics, 435–440

缺陷指标,437–440

defect metrics, 437–440

测量进度,435–437

measuring progress with, 435–437

概述, 435

overview of, 435

有用性,439–440

usefulness of, 439–440

迭代计划

Iteration planning

考虑所有观点,385-389

considering all viewpoints, 385–389

控制工作量,393

controlling workload, 393

学习项目细节,384–385

learning project details, 384–385

概述, 383–384

overview of, 383–384

写任务卡,389–392

writing task cards, 389–392

迭代评审会议,415

Iteration review meeting, 415

迭代结束,443–451

Iteration wrap up, 443–451

庆祝成功,449–451

celebrating successes, 449–451

迭代演示,443–444

demo of iteration, 443–444

改进的想法,447-449

ideas for improvement, 447–449

回顾,444-445

retrospectives, 444–445

回顾的“开始、停止、继续”练习,445–447

“start, stop, continue” exercise for retrospectives, 445–447

ITIL(信息技术基础设施库),90-91

ITIL (Information Technology Infrastructure Library), 90–91

J

行为举止,165

JBehave, 165

J控制台,234

JConsole, 234

JMeter

JMeter

性能基线测试,235

performance baseline tests, 235

性能测试, 223 , 234 , 313

performance testing, 223, 234, 313

JMS(Java 消息服务)

JMS (Java Messaging Service)

定义,502–503

definition, 502–503

与外部应用程序集成,243

integration with external applications and, 243

测试数据馈送和249

testing data feeds and, 249

J探查器,234

JProfiler, 234

J单元

JUnit

FitNesse 作为 TDD 的替代品,299

FitNesse as alternative for TDD, 299

功能测试,176

functional testing, 176

负载测试工具,234–235

load testing tools, 234–235

单元测试工具126、165、291 _

unit test tools, 126, 165, 291

JUnitPerf,234

JUnitPerf, 234

及时发展,369另见 预迭代活动

Just in time development, 369. See also Pre-iteration activities

K

关键成功因素

Key success factors

敏捷测试心态,482–483

agile testing mind-set, 482–483

自动化回归测试,484

automating regression testing, 484

大局观,490–491

big picture approach, 490–491

编码和测试作为一个过程,488–489

coding and testing as one process, 488–489

与客户合作,489–490

collaboration with customers, 489–490

持续集成 (CI),486–487

continuous integration (CI), 486–487

反馈,484–486

feedback, 484–486

核心实践基础,486

foundation of core practices, 486

增量方法(薄片,小块),488

incremental approach (thin slices, small chunks), 488

概述, 481

overview of, 481

实践之间的协同作用,489

synergy between practices, 489

技术债务管理,487–488

technical debt management, 487–488

测试环境,487

test environments, 487

整个团队的方法,482

whole team approach, 482

关键字驱动的测试,182–183

Keyword-driven tests, 182–183

国王,约瑟夫,176 岁

King, Joseph, 176

知识库,DTS,80-81

Knowledge base, DTS, 80–81

科尔乔纳森,201,204,211 _

Kohl, Jonathan, 201, 204, 211

König, Dierk, 320

König, Dierk, 320

大号

L

语言,共同需求,134–135

Language, need for common, 134–135

分层架构,116

Layered architecture, 116

精益测量,指标,74-75

Lean measurements, metrics, 74–75

学习

Learning

自动化策略和,303

automation strategy and, 303

持续改进原则,27

continuous improvement principle, 27

学习曲线、自动化和266–267 , 303

Learning curve, automation and, 266–267, 303

遗留代码,269

Legacy code, 269

遗留代码救援(Feathers),117

Legacy code rescue (Feathers), 117

遗留系统

Legacy systems

CCDE,269

ccde, 269

定义,502–503

definition, 502–503

记录错误和421

logging bugs and, 421

测试, 117

testing, 117

软件测试中的经验教训(Pettichord),485

Lessons Learned in Software Testing (Pettichord), 485

经验教训会议,383另见 回顾

Lessons learned sessions, 383. See also Retrospectives

轻量级流程,73–74

Lightweight processes, 73–74

轻量级测试计划,350

Lightweight test plans, 350

负载测试。请参见 性能和负载测试

Load testing. See Performance and load testing

负载运行器,234

LoadRunner, 234

负载测试,234

LoadTest, 234

物流,物理,65–66

Logistics, physical, 65–66

LogWatch 工具,212

LogWatch tool, 212

失去身份,QA 团队感到恐惧,44–45

Loss of identity, QA teams fearing, 44–45

卢维翁,克里斯托夫,63 岁

Louvion, Christophe, 63

M

可维护性测试,227–228

Maintainability testing, 227–228

管理,52–55

Management, 52–55

提高清晰度,373–374

advance clarity and, 373–374

文化变革,52–54

cultural change and, 52–54

概述, 52

overview of, 52

440提供指标

providing metrics to, 440

管理人员

Managers

52-54 岁的文化变革

cultural changes for, 52–54

如何影响测试,122–123

how to influence testing, 122–123

说经理的语言,55

speaking managerís language, 55

曼斯,玛丽·林恩,121–122

Manns, Mary Lynn, 121–122

手动测试

Manual testing

自动化与258–259

automation vs., 258–259

危险的, 289

peril of, 289

马尔卡诺,安东尼,83 岁426

Marcano, Antony, 83, 426

马里克、布赖恩5、24、97、134、170、203、303 _ _ _ _ _ _ _ _ _

Marick, Brian, 5, 24, 97, 134, 170, 203, 303

马丁·迈卡,169

Martin, Micah, 169

马丁·罗伯特·C.,169

Martin, Robert C., 169

矩阵

Matrices

高级测试和398–399

high-level tests and, 398–399

文本矩阵,350–353

text matrices, 350–353

马文,126 岁

Maven, 126

麦克马洪,克里斯,260

McMahon, Chris, 260

平均无故障时间,可靠性测试,230

Mean time between failure, reliability testing, 230

平均无故障时间,可靠性测试,230

Mean time to failure, reliability testing, 230

媒体,用于记录错误,423–424

Media, for logging bugs, 423–424

会议

Meetings

示威, 71 , 192

demonstrations, 71, 192

地理位置分散,376

geographically dispersed, 376

迭代开始,372

iteration kickoff, 372

迭代计划, 23–24 , 244 , 331 , 384 , 389

iteration planning, 23–24, 244, 331, 384, 389

迭代审查, 71 , 415

iteration review, 71, 415

预先规划,370–372

pre-planning, 370–372

发布计划, 338 , 345

release planning, 338, 345

回顾, 447

retrospective, 447

调度, 70

scheduling, 70

上浆过程和336–337

sizing process and, 336–337

站立, 177 , 429 , 462

standup, 177, 429, 462

团队参与,32

team participation and, 32

测试计划,263

test planning, 263

内存泄漏,237–238

Memory leaks, 237–238

内存管理测试,237–238

Memory management testing, 237–238

梅萨罗斯杰拉德、99、111、113、138、146、182、204、291、296、430 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Meszaros, Gerald, 99, 111, 113, 138, 146, 182, 204, 291, 296, 430

指标,74–79

Metrics, 74–79

代码覆盖率,360–364

code coverage, 360–364

77-78的沟通

communication of, 77–78

缺陷指标364-366,437-440

defect metrics, 364–366, 437–440

迭代指标,435–440

iteration metrics, 435–440

证明自动化投资的合理性,268

justifying investment in automation, 268

精益测量,74-75

lean measurements, 74–75

概述, 74

overview of, 74

通过测试,358–360

passing tests, 358–360

跟踪缺陷的原因, 52 , 75–77 , 82

reasons for tracking defects, 52, 75–77, 82

发布指标,358

release metrics, 358

投资回报率和78–79

ROI and, 78–79

不该做什么,77

what not to do with, 77

XP 雷达图,47–48

XP radar charts, 47–48

里程碑,庆祝成功,449–450

Milestones, celebrating successes, 449–450

MIME(多用途 Internet 邮件扩展)

MIME (Multipurpose Internet Mail Extensions)

定义,504

definition, 504

测试数据馈送和249

testing data feeds and, 249

思维导图,156–158

Mind maps, 156–158

心态

Mind-set

敏捷测试作为,20-21

agile testing as, 20–21

关键成功因素,482–483

key success factors, 482–483

主动,369–370

pro-active, 369–370

“迷你瀑布”现象,46-47

“Mini-waterfall” phenomenon, 46–47

模拟对象

Mock objects

定义,504

definition, 504

风险缓解和459

risk alleviation and, 459

实施工具,127

tools for implementing, 127

单元测试和,114

unit tests and, 114

模型

Mock-ups

促进沟通,430

facilitating communication and, 430

高级测试和398–399

high-level tests and, 398–399

故事和,380

stories and, 380

引出示例和要求的工具,160

tools for eliciting examples and requirements, 160

模型驱动开发,398

Model-driven development, 398

楷模

Models

质量模型,90-93

quality models, 90–93

UI 建模示例,399

UI modeling example, 399

监测工具, 212–213 , 235

Monitoring tools, 212–213, 235

多层方法,自动化策略,290-292

Multi-layered approach, automation strategy, 290–292

多用途 Internet 邮件扩展 (MIME)

Multipurpose Internet Mail Extensions (MIME)

定义,504

definition, 504

测试数据馈送和249

testing data feeds and, 249

N

命名约定,227

Naming conventions, 227

南特,126 岁

Nant, 126

导航、可用性测试和204

Navigation, usability testing and, 204

N行为,165

NBehave, 165

新负载,234

NeoLoad, 234

Nessus,漏洞扫描器,226

Nessus, vulnerability scanner, 226

.NET 内存分析器,234

.NET Memory Profiler, 234

网豆,125

NetBeans, 125

网络侦察兵,235

NetScout, 235

非功能测试。另见面 向技术的测试,评价产品(象限 4)

Non-functional testing. See also Technology-facing tests, critiquing the product (Quadrant 4)

交付产品和458–459

delivering product and, 458–459

功能测试比较,225

functional testing compared with, 225

要求,218–219

requirements, 218–219

什么时候执行,222

when to perform, 222

北,丹,165

North, Dan, 165

NSpec,165

NSpec, 165

N单元, 126 , 165

NUnit, 126, 165

O

Oleszkiewicz, 雅库布, 418

Oleszkiewicz, Jakub, 418

一次性测试,286–287

One-off tests, 286–287

开源工具

Open source tools

敏捷开源测试工具,172–175

agile open source test tools, 172–175

自动化和,314-315

automation and, 314–315

GUI 测试工具,172

GUI test tools, 172

IDE,124–125

IDEs, 124–125

打开网页加载,234

OpenWebLoad, 234

操作系统 (OS)、兼容性测试和230

Operating systems (OSs), compatibility testing and, 230

组织,37–44

Organizations, 37–44

敏捷开发的挑战,35

challenges of agile development, 35

冲突的文化,43

conflicting cultures, 43

客户关系和41–42

customer relationships and, 41–42

概述,37-38

overview of, 37–38

质量理念,38-40

quality philosophy, 38–40

大小和,42-43

size and, 42–43

可持续的测试速度,40-41

sustainable pace of testing and, 40–41

团队授权,44

team empowerment, 44

OS(操作系统)、兼容性测试和230

OSs (operating systems), compatibility testing and, 230

所有权,赋予团队所有权,50

Ownership, giving team ownership, 50

P

P

包装、产品交付和474–475

Packaging, product delivery and, 474–475

结对编程

Pair programming

代码审查和227

code review and, 227

受过培训的开发人员,61

developers trained in, 61

IDE 和125

IDEs and, 125

团队方法和,244

team approach and, 244

配对测试,413

Pair testing, 413

通过测试,发布指标,358–360

Passing tests, release metrics, 358–360

性能监视器,235

PerfMon, 235

Perforce, 124

Perforce, 124

性能和负载测试

Performance and load testing

自动化, 283

automating, 283

基线,235–237

baselines, 235–237

内存管理测试,237–238

memory management testing, 237–238

概述, 234

overview of, 234

产品交付和,458

product delivery and, 458

可扩展性测试,233–234

scalability testing, 233–234

测试环境,237

test environment, 237

工具,234–235

tools for, 234–235

什么时候执行,223

when to perform, 223

谁执行测试,220–221

who performs the test, 220–221

绩效、奖励和70–71

Performance, rewards and, 70–71

危险

Perils

忘记大局,148

forgetting the big picture, 148

素质警察心态,39

quality police mentality, 39

测试紧缩,416

the testing crunch, 416

等待周二的构建,280

waiting for Tuesdayís build, 280

你不是团队的一员,32

youíre not really part of the team, 32

珀金斯,史蒂夫,156 岁159 岁373 岁

Perkins, Steve, 156, 159, 373

Perl剪辑

PerlClip

数据生成工具,305

data generation tools, 305

用于生成测试数据的工具,212

tools for generating test data, 212

角色测试,202–204

Persona testing, 202–204

佩蒂索德布雷特175、264、485 _

Pettichord, Bret, 175, 264, 485

分阶段和封闭式开发73-74,129

Phased and gated development, 73–74, 129

实体物流,65–66

Physical logistics, 65–66

规划

Planning

前进, 43

advance, 43

迭代。请参见 迭代计划

iteration. See Iteration planning

发布/主题策划。请参阅 发布计划

release/theme planning. See Release planning

测试。请参见 测试计划

testing. See Test planning

PMO(项目管理办公室),440

PMO (Project Management Office), 440

波尔斯,安迪,134

Pols, Andy, 134

端口和适配器模式 (Cockburn),115

Ports and Adapters pattern (Cockburn), 115

开发后测试,467–468

Post-development testing, 467–468

迭代后错误,421

Post-iteration bugs, 421

庞德,234

Pounder, 234

三的力量

Power of Three

业务专家和,482

business expert and, 482

寻找共同语言,430

finding a common language, 430

良好的沟通和33 , 490

good communication and, 33, 490

解决问题,24

problem solving and, 24

解决观点分歧, 401 , 411

resolving differences in viewpoint, 401, 411

整个团队的方法和,482

whole team approach and, 482

务实的项目自动化260

Pragmatic Project Automation, 260

预迭代活动,369–382

Pre-iteration activities, 369–382

提前澄清,373

advance clarity, 373

提前制作故事的好处,370–372

benefits of working on stories in advance, 370–372

客户用同一种声音说话,373–374

customers speaking with one voice, 373–374

确定故事大小,375–376

determining story size, 375–376

评估所需的预先准备量,372–373

evaluating amount of advance preparation needed, 372–373

示例,378–380

examples, 378–380

收集有关需求的所有观点,374-375

gathering all viewpoints regarding requirements, 374–375

地理位置分散的团队,376–378

geographically dispersed team and, 376–378

概述, 369

overview of, 369

优先考虑缺陷,381

prioritizing defects, 381

积极主动的心态,369–370

pro-active mindset, 369–370

资源, 381

resources, 381

测试策略和,380–381

test strategies and, 380–381

预先计划会议,370-372

Pre-planning meeting, 370–372

原理、自动化

Principles, automation

敏捷编码实践,303–304

agile coding practices, 303–304

迭代方法,299–300

iterative approach, 299–300

保持简单,298–299

keep it simple, 298–299

边做边学,303

learning by doing, 303

概述, 298

overview of, 298

花时间做对,301-303

taking time to do it right, 301–303

整个团队方法,300–301

whole team approach, 300–301

原则,适用于敏捷测试人员

Principles, for agile testers

持续反馈,22

continuous feedback, 22

持续改进,27–28

continuous improvement, 27–28

勇气,25–26

courage, 25–26

为客户提供价值,22–23

delivering value to customer, 22–23

享受, 31

enjoyment, 31

面对面交流,23-25

face-to-face communication, 23–25

保持简单,26–27

keeping it simple, 26–27

概述,21-22

overview of, 21–22

以人为本,30

people focus, 30

响应变化,28-29

responsive to change, 28–29

自组织,29-30

self-organizing, 29–30

确定缺陷的优先​​级,381

Prioritizing defects, 381

优先报道,338-340

Prioritizing stories, 338–340

积极主动的心态,369–370

Pro-active mindset, 369–370

产品

Product

商业价值,31–33

business value, 31–33

送货。请参阅 交付产品

delivery. See Delivering product

批判性测试(Q3 和 Q4),101–104

tests that critique (Q3 & Q4), 101–104

是什么造就了产品,453–455

what makes a product, 453–455

产品拥有者

Product owner

在迭代计划中考虑所有观点,386–389

considering all viewpoints during iteration planning, 386–389

定义,504

definition, 504

迭代计划和,384

iteration planning and, 384

Scrum 角色, 141 , 373

Scrum roles, 141, 373

适合的工具,134

tools geared to, 134

生产

Production

记录错误和421

logging bugs and, 421

支持, 475

support, 475

生产代码

Production code

自动化测试金字塔和,277–278

automation test pyramid and, 277–278

定义,504

definition, 504

70 人提供价值

delivering value to, 70

程序员写作,48

programmers writing, 48

源代码控制和434

source code control and, 434

与测试同步,322

synchronization with testing, 322

测试先行的开发和,113

test-first development and, 113

测试支持,303–304

tests supporting, 303–304

类生产数据、自动化数据库和309–310

Production-like data, automating databases and, 309–310

职业发展,57

Professional development, 57

分析工具,234

Profiling tools, 234

程序员

Programmers

对自动化的态度,265–266

attitude regarding automation, 265–266

大图测试,397

big picture tests, 397

合作,413-414

collaboration with, 413–414

在迭代规划期间考虑所有观点,387–389

considering all viewpoints during iteration planning, 387–389

促进沟通,429-430

facilitating communication and, 429–430

使用400–401审查高级测试

reviewing high-level tests with, 400–401

测试人员与开发人员的比例,66–67

tester-developer ratio, 66–67

测试人员与45相比

testers compared with, 4, 5

培训, 61

training, 61

写任务卡和,391

writing task cards and, 391

项目管理办公室 (PMO),440

Project Management Office (PMO), 440

项目,PAS 示例,176–177

Projects, PAS example, 176–177

原型

Prototypes

作为通用语言可访问,134

accessible as common language, 134

模型和160

mock-ups and, 160

纸, 22 , 138–139 , 380 , 400 , 414

paper, 22, 138–139, 380, 400, 414

纸与绿野仙踪类型,275

paper vs. Wizard of Oz type, 275

UI(用户界面),107

UI (user interface), 107

脉冲,126

Pulse, 126

Python 的 PyUnit 单元测试工具,126

PyUnit unit test tool for Python, 126

Q

QA(质量保证)

QA (quality assurance)

定义,504

definition, 504

在职称中,31

in job titles, 31

独立QA团队,60人

independent QA team, 60

可与“测试”互换,59

interchangeable with “test,” 59

整个团队的方法,39

whole team approach, 39

在传统团队中工作,9

working on traditional teams, 9

象限 1。参见 面向技术的测试,支持团队(象限 1)

Quadrant 1. See Technology-facing tests, supporting team (Quadrant 1)

象限 2。参见 面向业务的测试,支持团队(象限 2)

Quadrant 2. See Business-facing tests, supporting team (Quadrant 2)

第 3 象限。参见 面向业务的测试,评价产品(第 3 象限)

Quadrant 3. See Business-facing tests, critiquing the product (Quadrant 3)

第 4 象限。参见 面向技术的测试,评价产品(第 4 象限)

Quadrant 4. See Technology-facing tests, critiquing the product (Quadrant 4)

象限

Quadrants

自动化测试类别,274–276

automation test categories, 274–276

面向业务(Q2 和 Q3),97–98

business facing (Q2 & Q3), 97–98

上下文驱动的测试和,106-108

context-driven testing and, 106–108

批评产品(Q3 和 Q4),104

critiquing the product (Q3 & Q4), 104

管理技术债务,106

managing technical debt, 106

概述, 97–98

overview of, 97–98

作为规划指南,490

as planning guide, 490

测试的目的和,97

purpose of testing and, 97

第 1 象限摘要,99

Quadrant 1 summary, 99

第 2 象限摘要,99–100

Quadrant 2 summary, 99–100

第 3 象限摘要,101–102

Quadrant 3 summary, 101–102

象限 4 摘要,102–104

Quadrant 4 summary, 102–104

共同责任和105–106

shared responsibility and, 105–106

故事完成,104-105

story completion and, 104–105

支持团队(Q1 和 Q2),100–101

supporting the team (Q1 & Q2), 100–101

面向技术(Q1 和 Q4),97–98

technology facing (Q1 & Q4), 97–98

质量

Quality

客户在制定质量标准中的作用,26

customer role in setting quality standards, 26

型号,90–93

models, 90–93

关于38-40 的组织理念

organizational philosophy regarding, 38–40

质量保证。参见 QA(质量保证)

Quality assurance. See QA (quality assurance)

素质警察心态,57

Quality police mentality, 57

问题,引出需求,135-136

Questions, for eliciting requirements, 135–136

R

R

雷达图,XP,47–48

Radar charts, XP, 47–48

乔纳森·拉斯穆森,11 岁

Rasmusson, Jonathan, 11

录制/回放工具

Record/playback tools

自动化策略和, 294 , 296–297

automation strategy and, 294, 296–297

GUI 测试工具,171–172

GUI test tools, 171–172

恢复测试,459

Recovery testing, 459

冗余测试,232

Redundancy tests, 232

大卫·里德171 , 377

Reed, David, 171, 377

重构

Refactoring

定义,504

definition, 504

支持的 IDE,124–126

IDEs supporting, 124–126

回归套房,434

Regression suite, 434

回归测试,432–434

Regression tests, 432–434

作为安全网的自动回归测试,261–262

automated regression tests as a safety net, 261–262

自动化作为成功因素,484

automating as success factor, 484

检查大图,434

checking big picture, 434

定义,504

definition, 504

探索性测试和212

exploratory testing and, 212

保持构建“绿色”,433

keeping the build “green,” 433

保持快速构建,433–434

keeping the build quick, 433–434

记录错误和420

logging bugs and, 420

回归套房和,434

regression suite and, 434

发布候选人和458

release candidates and, 458

发布

Release

验收标准,470–473

acceptance criteria, 470–473

游戏结束, 327 , 456–457

end game, 327, 456–457

管理, 474

management, 474

产品交付,470

product delivery, 470

如果还没有准备好怎么办,463–464

what if it is not ready, 463–464

发布候选人

Release candidates

具有挑战性的候选发布版本,473

challenging release candidate builds, 473

定义,505

definition, 505

测试, 458

testing, 458

发布指标

Release metrics

代码覆盖率,360–364

code coverage, 360–364

缺陷指标,364–366

defect metrics, 364–366

概述, 358

overview of, 358

通过测试,358–360

passing tests, 358–360

发行说明,474

Release notes, 474

发布计划,329–367

Release planning, 329–367

概述, 329

overview of, 329

优先考虑,338–340

prioritizing and, 338–340

目的,330-331

purpose of, 330–331

范围,340–344

scope, 340–344

尺寸和332–337

sizing and, 332–337

测试计划备选方案,350–354

test plan alternatives, 350–354

测试计划,345–350

test planning, 345–350

能见度和354–366

visibility and, 354–366

可靠性测试

Reliability testing

概述,230-231

overview of, 230–231

远程数据监控系统示例,250–251

Remote Data Monitoring system example, 250–251

远程数据监控系统示例

Remote Data Monitoring system example

验收测试,245

acceptance tests, 245

应用程序,242–243

application, 242–243

应用测试象限,252–253

applying test quadrants, 252–253

自动化功能测试结构,245–247

automated functional test structure, 245–247

记录测试代码,251

documenting test code, 251

嵌入式测试,248

embedded testing, 248

端到端测试,249–250

end-to-end tests, 249–250

探索性测试,248–249

exploratory testing, 248–249

概述, 242

overview of, 242

可靠性测试,250–251

reliability testing, 250–251

报告测试结果,251

reporting test results, 251

团队和流程,243–244

team and process, 243–244

测试数据源,249

testing data feeds, 249

单元测试,244–245

unit tests, 244–245

用户验收测试,250

user acceptance testing, 250

网络服务,247-248

web services, 247–248

远程团队成员。查看 地理上分散的团队

Remote team member. See Geographically dispersed teams

重复性任务,自动化,284

Repetitive tasks, automating, 284

报告

Reports

文档和208–210

documentation and, 208–210

远程数据监控系统示例,251

Remote Data Monitoring system example, 251

存储库,124

Repository, 124

要求

Requirements

面向业务的测试寻址,130

business-facing tests addressing, 130

402的文档

documentation of, 402

收集有关需求的所有观点,374-375

gathering all viewpoints regarding requirements, 374–375

如何引出,135-140

how to elicit, 135–140

无功能,218–219

nonfunctional, 218–219

窘境,132–134

quandary, 132–134

引出示例和要求的工具,155-156

tools for eliciting examples and requirements, 155–156

资源

Resources

完成故事,381

completing stories and, 381

招聘敏捷测试员,67-69

hiring agile tester, 67–69

概述, 66

overview of, 66

测试人员与开发人员的比例,66–67

tester-developer ratio, 66–67

测试和434–435

testing and, 434–435

响应时间

Response time

原料药,411

API, 411

负载测试和234–235

load testing and, 234–235

可衡量的目标,76

measurable goals and, 76

网络服务和207

web services and, 207

回顾

Retrospectives

持续改进,28

continuous improvement and, 28

改进的想法,447-449

ideas for improvement, 447–449

迭代计划和,383

iteration planning and, 383

概述, 444–445

overview of, 444–445

流程改进,90

process improvement and, 90

“开始、停止和继续”练习,445–447

“start, stop, and continue” exercise, 445–447

投资回报。请参见 ROI(投资回报率)

Return on investment. See ROI (return on investment)

奖励、绩效和,70–71

Rewards, performance and, 70–71

富客户端单元测试工具,127

Rich-client unit testing tools, 127

瑞星,琳达,121–122

Rising, Linda, 121–122

风险

Risk

风险分析, 198 , 286 , 290 , 345–346

risk analysis, 198, 286, 290, 345–346

风险评估,407-409

risk assessment, 407–409

测试缓解,147-149

test mitigating, 147–149

罗杰斯, 保罗, 242 , 310 , 388 , 398

Rogers, Paul, 242, 310, 388, 398

ROI(投资回报率)

ROI (return on investment)

自动化和, 264

automation and, 264

定义,505

definition, 505

精益测量和,75

lean measurement and, 75

指标和,78-79

metrics and, 78–79

说经理的语言,55

speaking managerís language, 55

角色、功能、商业价值模式,155

Role, function, business value pattern, 155

角色

Roles

冲突或多重角色,45

conflicting or multiple roles, 45

48-49岁之间的文化差异

cultural differences among, 48–49

客户团队,7

customer team, 7

开发团队,7–8

developer team, 7–8

互动,8

interaction of, 8

RPG单位, 118

RPGUnit, 118

R规格, 165 , 318

RSpec, 165, 318

Ruby 测试::单元, 170

Ruby Test::Unit, 170

Ruby 与 Watir

Ruby with Watir

功能测试,247

functional testing, 247

GUI 测试,285

GUI testing, 285

识别缺陷,212

identifying defects with, 212

用于自动化测试的关键字或操作词,182

keywords or actions words for automating tests, 182

概述,172-174

overview of, 172–174

测试自动化,186

test automation with, 186

红宝石模拟,127

RubyMock, 127

规则、错误管理和425

Rules, managing bugs and, 425

小号

S

安全测试,232

Safety tests, 232

桑托斯,拉斐尔,448

Santos, Rafael, 448

满意条件。查看 满意条件

Satisfaction conditions. See Conditions of satisfaction

可扩展性测试,233–234

Scalability testing, 233–234

情景测试,192–193

Scenario testing, 192–193

流程图和,194-195

flow diagrams and, 194–195

概述,192-195

overview of, 192–195

肥皂剧测试,193

soap opera tests, 193

范围,340–344

Scope, 340–344

面向业务的测试定义,134

business-facing tests defining, 134

截止日期和时间表,以及340–341

deadlines and timelines and, 340–341

关注价值,341-342

focusing on value, 341–342

概述, 340

overview of, 340

全系统影响,342

system-wide impact, 342

测试计划和345

test plans and, 345

第三方参与,342-344

third-party involvement and, 342–344

范围蠕变385,412

Scope creep, 385, 412

脚本

Scripts

自动比较,283

automating comparisons, 283

作为自动化工具,297

as automation tools, 297

转换脚本,461

conversion scripts, 461

数据生成工具,305

data generation tools, 305

探索性测试和211–212

exploratory testing and, 211–212

敏捷

Scrum

产品负责人角色141,373

product owner role, 141, 373

远程数据监控系统示例,244

Remote Data Monitoring system example, 244

冲刺评论,444

sprint reviews, 444

ScrumMaster

ScrumMaster

流程改进方法,448–449

approach to process improvement, 448–449

调整故事大小,336–337

sizing stories and, 336–337

写任务卡和,391

writing task cards and, 391

SDD(故事测试驱动开发)

SDD (story test-driven development)

识别变异,410

identifying variations, 410

概述,262–263

overview of, 262–263

测试先行的开发和,263

test-first development and, 263

测试 Web 服务和170

testing web services and, 170

安全测试

Security testing

攻击者由外向内的方法,225

outside-in approach of attackers, 225

概述,223-227

overview of, 223–227

所需的专业知识,220

specialized knowledge required for, 220

Selenium

GUI 测试工具,174–175

GUI test tools, 174–175

实施自动化,316–318

implementing automation, 316–318

开源工具,163

open source tools, 163

测试自动化186,316

test automation with, 186, 316

自组织

Self-organization

原则,29-30

principles, 29–30

自组织团队,69

self-organizing teams, 69

基于会话的测试,200–201

Session-based testing, 200–201

设置

Setup

自动化,284–285

automating, 284–285

探索性测试,211–212

exploratory testing, 211–212

共享资源

Shared resources

访问, 43

access to, 43

专家作为,301

specialists as, 301

写作任务和,390

writing tasks and, 390

共同责任,105–106

Shared responsibility, 105–106

Shout-Out 鞋盒,450

Shout-Out Shoebox, 450

“给我看”,与程序员合作,413–414

“Show me,” collaboration with programmers, 413–414

简单

Simplicity

自动化和298–299

automation and, 298–299

编码,406

coding, 406

记录错误和428–429

logging bugs and, 428–429

“保持简单”的原则,26-27

principle of “keeping it simple,” 26–27

模拟器工具

Simulator tools

嵌入式测试和,248

embedded testing and, 248

概述, 213

overview of, 213

规模,组织性,42-43

Size, organizational, 42–43

上浆故事,332–337

Sizing stories, 332–337

例如,334–337

example of, 334–337

如何,332–333

how to, 332–333

概述, 332

overview of, 332

测试人员在333–334中的角色

tester’s role in, 333–334

技能

Skills

适应性和,39-40

adaptability and, 39–40

与态度,20

vs. attitude, 20

持续改进原则,27

continuous improvement principle, 27

谁执行测试,以及220–221

who performs tests and, 220–221

小块,增量开发,144-146

Small chunks, incremental development, 144–146

肥皂

SOAP

定义,505

definition, 505

性能测试和,223,234

performance tests and, 223, 234

肥皂剧测试,193

Soap opera tests, 193

肥皂用户界面

soapUI

定义,505

definition, 505

性能测试和,223,234

performance tests and, 223, 234

测试 Web 服务,170–171

testing Web Services, 170–171

SOA测试,234

SOATest, 234

基于软件的工具,163

Software-based tools, 163

软件配置管理模式:有效的团队合作、实用的集成(Berczuk 和 Appleton),124

Software Configuration Management Patterns: Effective Teamwork, Practical Integrations (Berczuk and Appleton), 124

软件残局(盖伦),471

Software Endgames (Galen), 471

源代码控制

Source code control

的好处, 255

benefits of, 255

概述,123-124

overview of, 123–124

工具124,320 _

tools for, 124, 320

SOX 合规性,469

SOX compliance, 469

用同一个声音说话,客户,373-374

Speak with one voice, customers, 373–374

专业化,220-221

Specialization, 220–221

以速度为目标,112

Speed as a goal, 112

峰值、开发和测试,381

Spikes, development and test, 381

电子表格

Spreadsheets

测试电子表格,353

test spreadsheets, 353

引出示例和要求的工具,159

tools for eliciting examples and requirements, 159

Sprint 评论,444另请参阅演示/演示

Sprint reviews, 444. See also Demos/demonstrations

SQL* 加载程序,460

SQL*Loader, 460

稳定性测试,28

Stability testing, 28

暂存环境,458

Staging environment, 458

站立会议, 177 , 429 , 462

Stand-up meetings, 177, 429, 462

标准

Standards

可维护性和,227

maintainability and, 227

质量模型和,90-93

quality models and, 90–93

“开始、停止、继续”练习,回顾,445-447

“Start, stop, continue” exercise, retrospectives, 445–447

静态分析,安全测试工具,225

Static analysis, security testing tools, 225

钢螺纹,增量开发144、338、345

Steel thread, incremental development, 144, 338, 345

故事。另请参阅 面向业务的测试

Stories. See also Business-facing tests

在迭代之前工作的好处,370–372

benefits of working on in advance of iterations, 370–372

129–130的简要说明

briefness of, 129–130

面向业务的测试,130

business-facing tests as, 130

确定故事大小,375–376

determining story size, 375–376

编码时专注于一个故事,411-412

focusing on one story when coding, 411–412

识别变异,410

identifying variations, 410

知道故事何时结束,104–105

knowing when a story is done, 104–105

记录错误和420–421

logging bugs and, 420–421

模型和380

mock-ups and, 380

优先排序,338–340

prioritizing, 338–340

资源和,381

resources and, 381

范围和,340

scope and, 340

浆纱。查看 上浆故事

sizing. See Sizing stories

简单开始,133,406

starting simple, 133, 406

定义的故事测试,505

story tests defined, 505

全系统影响,342

system-wide impact of, 342

测试计划和345

test plans and, 345

测试策略和,380–381

test strategies and, 380–381

可测试,393–396

testable, 393–396

将错误视为425

treating bugs as, 425

故事板

Story boards

燃尽图,429

burndown charts, 429

定义,505–506

definition, 505–506

示例,356–357

examples, 356–357

在线, 357 , 384

online, 357, 384

物理, 356

physical, 356

贴纸和,355

stickers and, 355

任务, 222 , 355 , 436

tasks, 222, 355, 436

虚拟, 357 , 384 , 393

virtual, 357, 384, 393

正在进行的工作,390

work in progress, 390

故事卡

Story cards

审计和, 89

audits and, 89

处理错误和424–425

dealing with bugs and, 424–425

迭代计划和,244

iteration planning and, 244

故事叙述,409

story narrative on, 409

故事测试驱动开发。参见 SDD(故事测试驱动开发)

Story test-driven development. See SDD (story test-driven development)

Strangler 应用程序 (Fowler),116–117

Strangler application (Fowler), 116–117

战略

Strategy

自动化。请参阅 自动化策略

automation. See Automation strategy

测试计划与测试策略,86-87

test planning vs. test strategy, 86–87

测试策略,380–381

test strategies, 380–381

策略,用于编写测试

Strategy, for writing tests

逐步构建测试,178–179

building tests incrementally, 178–179

迭代计划和,372

iteration planning and, 372

保持测试通过,179

keep the tests passing, 179

概述,177-178

overview of, 177–178

测试设计模式,179–183

test design patterns, 179–183

可测性和,183–185

testability and, 183–185

压力测试。请参见 负载测试

Stress testing. See Load testing

颠覆 (SVN), 124 , 320

Subversion (SVN), 124, 320

成功因素。查看 关键成功因素

Success factors. See Key success factors

成功,庆祝

Successes, celebrating

改变实施,50-52

change implementation and, 50–52

迭代总结和,449-451

iteration wrap up and, 449–451

萨姆雷尔,梅根,365 岁450 岁

Sumrell, Megan, 365, 450

可持续步伐,测试40–41,303

Sustainable pace, of testing, 40–41, 303

SVN (颠覆124,320

SVN (Subversion), 124, 320

SWTBot GUI测试工具,127

SWTBot GUI test tool, 127

实践之间的协同作用,489

Synergy, between practices, 489

系统,故事的全系统影响,342

System, system-wide impact of story, 342

T

tail-f, 212

tail-f, 212

塔尔塔利亚科尼439、454、470、473 _ _ _

Tartaglia, Coni, 439, 454, 470, 473

任务板。查看 故事板

Task boards. See Story boards

任务卡

Task cards

自动化测试和394–395

automating testing and, 394–395

迭代计划和389–392

iteration planning and, 389–392

产品交付和462–463

product delivery and, 462–463

任务

Tasks

完成测试任务,415–416

completing testing tasks, 415–416

定义,505–506

definition, 505–506

TDD(测试驱动开发)

TDD (test-driven development)

自动驾驶测试,262–263

automated tests driving, 262–263

缺陷和490

defects and, 490

定义,506

definition, 506

概述,5

overview of, 5

测试优先开发与113–114相比

Test-First Development compared with, 113–114

单元测试和111 , 244–245

unit tests and, 111, 244–245

团队城市,126

Team City, 126

团队结构,59–65

Team structure, 59–65

敏捷项目团队,64–65

agile project teams, 64–65

独立QA团队,60人

independent QA team, 60

将测试人员集成到敏捷项目中,61–63

integration of testers into agile project, 61–63

概述, 59

overview of, 59

传统职能结构与敏捷结构,64

traditional functional structure vs agile structure, 64

团队

Teams

自动化作为团队的努力,484

automation as team effort, 484

建筑,69–71

building, 69–71

庆祝成功,50–52

celebrating success, 50–52

同地办公,65–66 岁

co-located, 65–66

控制工作量和393

controlling workload and, 393

顾客,7

customer, 7

开发人员,7–8

developer, 7–8

赋权,44

empowerment of, 44

促进沟通,429-432

facilitating communication and, 429–432

地理分散, 376–378 , 431–432

geographically dispersed, 376–378, 431–432

给予所有团队成员同等的权重,31

giving all team members equal weight, 31

给予所有权,50

giving ownership to, 50

67-69 岁招聘敏捷测试员

hiring agile tester for, 67–69

客户和开发团队之间的互动,8

interaction between customer and developer teams, 8

迭代计划和384–385

iteration planning and, 384–385

物流, 59

logistics, 59

解决问题和,123

problem solving and, 123

远程数据监控系统示例,243–244

Remote Data Monitoring system example, 243–244

共同责任和105–106

shared responsibility and, 105–106

传统的,9-10

traditional, 9–10

使用测试支持象限12,100–101

using tests to support Quadrants 1 and 2, 100–101

整个团队的方法。查看 整个团队方法

whole team approach. See Whole team approach

在敏捷团队中工作,10-12

working on agile teams, 10–12

拆解,用于测试,307–308

Teardown, for tests, 307–308

技术债务

Technical debt

缺陷,418

defects as, 418

定义,506

definition, 506

管理, 106 , 487–488

managing, 106, 487–488

面向技术的测试

Technology-facing tests

概述,5

overview of, 5

象限 1 & 4, 97–98

Quadrants 1 & 4, 97–98

面向技术的测试,批评产品(第 4 象限),217–239

Technology-facing tests, critiquing the product (Quadrant 4), 217–239

基线,235–237

baselines, 235–237

编码和测试,以及412–413

coding and testing and, 412–413

兼容性测试,229–230

compatibility testing, 229–230

安装性测试,231-232

installability testing, 231–232

互操作性测试,228–229

interoperability testing, 228–229

可维护性测试,227-228

maintainability testing, 227–228

内存管理测试,237–238

memory management testing, 237–238

概述,217-219

overview of, 217–219

性能和负载测试,234

performance and load testing, 234

性能和负载测试工具,234-235

performance and load testing tools, 234–235

可靠性测试, 230–231 , 250–251

reliability testing, 230–231, 250–251

可扩展性测试,233–234

scalability testing, 233–234

安全测试,223-227

security testing, 223–227

测试环境和,237

test environment and, 237

何时使用,222–223

when to use, 222–223

谁执行测试,220–222

who performs the test, 220–222

面向技术的测试,支持团队(象限 1)

Technology-facing tests, supporting team (Quadrant 1)

构建工具,126

build tools, 126

设计时考虑到测试,115–118

designing with testing in mind, 115–118

轻松完成任务,114–115

ease of accomplishing tasks, 114–115

IDE,124–126

IDEs for, 124–126

基础设施支持,111-112

infrastructure supporting, 111–112

概述,109-110

overview of, 109–110

目的,110–111

purpose of, 110–111

源代码控制,123–124

source code control, 123–124

速度的好处,112–114

speed as benefit of, 112–114

及时反馈,118-119

timely feedback, 118–119

工具包,123

toolkit for, 123

单元测试工具,126–127

unit test tools, 126–127

单元测试,244–245

unit tests, 244–245

如果团队不执行这些测试怎么办,121–123

what to do if team doesn’t perform these tests, 121–123

在哪里/何时停止,119–121

where/when to stop, 119–121

测试自动化金字塔

Test automation pyramid

多层自动化方法,290-291

multi-layered approach to automation and, 290–291

概述,276–279

overview of, 276–279

三只小猪的比喻,278

three little pigs metaphor, 278

UI 背后的测试,282

Test behind UI, 282

测试用例

Test cases

增加复杂性,407

adding complexity, 407

作为文档,402

as documentation, 402

示例驱动开发,379

example-driven development, 379

识别变异,410

identifying variations, 410

从简单开始,406

starting simple, 406

测试覆盖率(和/或代码覆盖率),360–364

Test coverage (and/or code coverage), 360–364

测试设计模式,179–183

Test design patterns, 179–183

构建/操作/检查模式,180

Build/Operate/Check pattern, 180

数据驱动和关键字驱动的测试,182–183

data-driven and keyword-driven tests, 182–183

概述, 179

overview of, 179

测试起源模式(Veragen),179

test genesis patterns (Veragen), 179

基于时间的活动和事件模式,181

time-based, activity, and event patterns, 181

测试双打

Test doubles

定义,506

definition, 506

分层架构和,116

layered architectures and, 116

测试驱动开发。请参见 TDD(测试驱动开发)

Test-driven development. See TDD (test-driven development)

测试环境, 237 , 487

Test environments, 237, 487

测试先行开发

Test-First Development

定义,506

definition, 506

与 TDD(测试驱动开发)相比,113–114

TDD (test-driven development) compared with, 113–114

测试管理,186

Test management, 186

测试管理工具包(象限 2),186

Test management toolkit (Quadrant 2), 186

测试计划备选方案,350 -354

Test plan alternatives, 350-354

测试计划,345–350

Test planning, 345–350

自动化测试列表,测试计划备选方案,353–354

automated test lists, test plan alternatives, 353–354

基础设施和346–347

infrastructure and, 346–347

概述, 86 , 345

overview of, 86, 345

写作的原因,345-346

reasons for writing, 345–346

测试环境和,347–348

test environment and, 347–348

测试计划备选方案,350–354

test plan alternatives, 350–354

测试计划,轻量级350

test plans, lightweight 350

测试计划样本,351

test plan sample, 351

测试策略对比,86-88

test strategy vs., 86–88

可追溯性和88

traceability and, 88

测试类型和,346

types of tests and, 346

从哪里开始,345

where to start, 345

检测结果

Test results

沟通,357–358

communicating, 357–358

组织,322–324

organizing, 322–324

发布计划和349–350

release planning and, 349–350

测试技能。查看 技能

Test skills. See Skills

测试钉,381

Test spikes, 381

测试电子表格,353

Test spreadsheets, 353

测试策略

Test strategy

迭代、预迭代活动和380–381

iterations, pre-iteration activities and, 380–381

测试计划对比,86-88

test plan vs., 86–88

测试存根

Test stubs

定义,506

definition, 506

与外部应用程序集成,以及459

integration with external applications and, 459

单元测试和,127

unit tests and, 127

测试团队,506–507另见 团队

Test teams, 506–507. See also Teams

测试工具。另见 工具包

Test tools. See also Toolkits

API 层功能,168–170

API-layer functional, 168–170

探索性测试,210–211

exploratory testing, 210–211

生成测试数据,212

generating test data with, 212

GUI 测试,170–176

GUI tests, 170–176

自制, 175

home-brewed, 175

自产, 314

home-grown, 314

IDE,124–126

IDEs, 124–126

性能测试,234–235

performance testing, 234–235

安全测试,225

security testing, 225

单元级测试126-127,165-168

unit-level tests, 126–127, 165–168

网络服务测试,170

web service tests, 170

测试类型

Test types

阿尔法/贝塔,466–467

alpha/beta, 466–467

探索性的。参见 探索性测试 (ET)

exploratory. See Exploratory testing (ET)

功能性的。请参见 功能测试

functional. See Functional testing

图形用户界面。请参见 GUI 测试

GUI. See GUI testing

整合, 229 , 459

integration, 229, 459

加载。请参见 负载测试

load. See Load testing

表现。请参见 性能和负载测试

performance. See Performance and load testing

可靠性, 230–231 , 250–251

reliability, 230–231, 250–251

安全, 220 , 223–227

security, 220, 223–227

压力。请参见 负载测试

stress. See Load testing

单元。请参见 单元测试

unit. See Unit testing

可用性。参见 可用性测试

usability. See Usability testing

用户验收测试。参见 UAT(用户验收测试)

user acceptance testing. See UAT (user acceptance testing)

测试写作策略。参见 Strategy,用于编写测试

Test writing strategy. See Strategy, for writing tests

可测试性,183–185

Testability, 183–185

自动与手动象限 2 测试,185

automated vs. manual Quadrant 2 tests, 185

自动化和149–150

automation and, 149–150

代码设计和测试设计,184–185

code design and test design and, 184–185

概述, 183

overview of, 183

故事数,393–396

of stories, 393–396

测试员

Testers

增加价值,12

adding value, 12

敏捷测试人员, 4 , 19–20

agile testers, 4, 19–20

敏捷测试心态,20-21

agile testing mindset, 20–21

自动化允许专注于更重要的工作,260

automation allowing focus on more important work, 260

与客户合作,396–397

collaboration with customers, 396–397

在迭代计划中考虑所有观点,386–389

considering all viewpoints during iteration planning, 386–389

控制工作量和393

controlling workload and, 393

定义,507

definition, 507

促进沟通,429-430

facilitating communication, 429–430

反馈和,486

feedback and, 486

招聘敏捷测试员,67-69

hiring agile tester, 67–69

如何影响测试,121–122

how to influence testing, 121–122

将测试人员集成到敏捷项目中,61–63

integration of testers into agile project, 61–63

迭代和,327

iterations and, 327

让工作更轻松,114–115

making job easier, 114–115

上浆故事,333–334

sizing stories, 333–334

测试人员与开发人员的比例,66–67

tester-developer ratio, 66–67

写任务卡和,391

writing task cards and, 391

测试员的权利法案,49–50

Tester's bill of rights, 49–50

测试

Testing

同时编码和测试,409–410

coding and testing simultaneously, 409–410

完成测试任务,415–416

completing testing tasks, 415–416

识别变异,410

identifying variations, 410

管理,320-322

managing, 320–322

组织测试结果,322–324

organizing test results, 322–324

组织测试,319–322

organizing tests, 319–322

计划时间,455–456

planning time for, 455–456

开发后周期,467–468

post-development cycles, 467–468

象限。象限

quadrants. See Quadrants

发布候选人,458

release candidates, 458

风险评估和407–409

risk assessment and, 407–409

可持续步伐,40-41

sustainable pace of, 40–41

传统与敏捷,12–15

traditional vs. agile, 12–15

测试的透明度,321-322

transparency of tests, 321–322

在上下文中测试

Testing in context

上下文驱动的测试和,106-108

context-driven testing and, 106–108

定义,502

definition, 502

TestNG GUI测试工具,127

TestNG GUI test tool, 127

永不失败的测试,286

Tests that never fail, 286

文本矩阵,350–353

Text matrices, 350–353

研磨机,234

The Grinder, 234

主题。另请参阅 发布计划

Themes. See also Release planning

定义,507

definition, 507

优先报道,339

prioritizing stories and, 339

写任务卡和,392

writing task cards and, 392

薄片、增量开发和338

Thin slices, incremental development and, 338

第三方

Third parties

兼容性测试和230

compatibility testing and, 230

发布计划和342–344

release planning and, 342–344

软件, 163

software, 163

托尔森,迈克,203

Tholfsen, Mike, 203

托马斯,迈克,116 岁194 岁

Thomas, Mike, 116, 194

三只小猪的比喻,278

Three little pigs metaphor, 278

时间表、范围和340–341

Timelines, scope and, 340–341

工具包(象限 1)

Toolkit (Quadrant 1)

构建工具,126

build tools, 126

IDE,124–126

IDEs, 124–126

概述, 123

overview of, 123

源代码控制,123–124

source code control, 123–124

单元测试工具,126–127

unit test tools, 126–127

工具包(象限 2)

Toolkit (Quadrant 2)

API 层功能测试工具,168–170

API-layer functional test tools, 168–170

自动化工具,164–165

automation tools, 164–165

逐步构建测试,178–179

building tests incrementally, 178–179

清单,156

checklists, 156

流程图,160-163

flow diagrams, 160–163

GUI 测试工具,170–176

GUI test tools, 170–176

保持测试通过,179

keep the tests passing, 179

思维导图,156–158

mind maps, 156–158

模型,160

mock-ups, 160

基于软件的工具,163

software-based tools, 163

电子表格,159

spreadsheets, 159

写作测试的策略,177-178

strategies for writing tests, 177–178

测试设计模式,179–183

test design patterns, 179–183

测试管理,186

test management, 186

可测性和,183–185

testability and, 183–185

工具策略,153–155

tool strategy, 153–155

引出示例和要求的工具,155-156

tools for eliciting examples and requirements, 155–156

单元级测试工具,165–168

unit-level test tools, 165–168

Web服务测试工具,170

Web service test tool, 170

工具包(第 3 象限)

Toolkit (Quadrant 3)

仿真器工具,213–214

emulator tools, 213–214

监控工具,212-213

monitoring tools, 212–213

模拟器工具,213

simulator tools, 213

用户验收测试,250

user acceptance testing, 250

工具包(第 4 象限)

Toolkit (Quadrant 4)

基线,235–237

baselines, 235–237

性能和负载测试工具,234-235

performance and load testing tools, 234–235

工具

Tools

API 层功能测试工具,168–170

API-layer functional test tools, 168–170

自动化,164–165

automation, 164–165

数据生成,304–305

data generation, 304–305

缺陷跟踪,83–85

defect tracking, 83–85

引出示例和要求155-156,159-163

eliciting examples and requirements, 155–156, 159–163

仿真器工具,213–214

emulator tools, 213–214

探索性测试,210–211

exploratory testing, 210–211

生成测试数据,212

generating test data, 212

GUI 测试工具,170–176

GUI test tools, 170–176

自制, 175

home-brewed, 175

自产, 314

home-grown, 314

IDE,124–126

IDEs, 124–126

负载测试,234–235

load testing, 234–235

监测,212-213

monitoring, 212–213

开源, 172 , 314–315

open source, 172, 314–315

性能测试,234–235

performance testing, 234–235

对于产品所有者和业务专家,134

for product owners and business experts, 134

安全测试,225

security testing, 225

模拟器, 213

simulators, 213

基于软件,163

software-based, 163

单元级测试126-127,165-168

unit-level tests, 126–127, 165–168

供应商/商业,315-316

vendor/commercial, 315–316

Web服务测试工具,170

web service test tool, 170

工具、自动化

Tools, automation

敏捷友好型,316

agile-friendly, 316

一次使用一个工具,312–313

applying one tool at a time, 312–313

自制, 175

home-brewed, 175

自产, 314

home-grown, 314

确定工具要求,311–312

identifying tool requirements, 311–312

开源,314–315

open source, 314–315

选择,294–298

selecting, 294–298

供应商,315–316

vendors, 315–316

可追溯性

Traceability

DTS 和82

DTS and, 82

矩阵,86

matrices, 86

测试计划和,88

test planning and, 88

跟踪、测试任务和状态,354–357

Tracking, test tasks and status, 354–357

传统流程,转型。请参见将传统流程转变为敏捷流程

Traditional processes, transitioning. See Transitioning traditional processes to agile

传统球队,9–10

Traditional teams, 9–10

传统与敏捷测试,12–15

Traditional vs. agile testing, 12–15

训练

Training

作为可交付成果,469

as deliverable, 469

缺乏, 45

lack of, 45

将传统流程转变为敏捷流程,73–93

Transitioning traditional processes to agile, 73–93

缺陷跟踪。请参见 缺陷跟踪

defect tracking. See Defect tracking

现有流程和,88–92

existing process and, 88–92

精益测量,74-75

lean measurements, 74–75

轻量级流程和,73–74

lightweight processes and, 73–74

指标和,74-79

metrics and, 74–79

概述, 73

overview of, 73

测试计划。请参见 测试计划

test planning. See Test planning

ü

U

UAT(用户验收测试)

UAT (user acceptance testing)

开发后测试周期,467–468

post-development testing cycles, 467–468

产品交付和464–466

product delivery and, 464–466

在象限 3, 102

in Quadrant 3, 102

发布计划331,346

release planning for, 331, 346

远程数据监控系统示例,250

Remote Data Monitoring system example, 250

在测试计划中,351

in test plan, 351

尝试新功能,102

tryng out new features and, 102

在迭代启动会议上写作,372

writing at iteration kickoff meeting, 372

UI(用户界面)。另见GUI(图形用户界面)

UI (user interface). See also GUI (graphical user interface)

自动化策略和,293

automation strategy and, 293

建模和, 399

modeling and, 399

单元测试工具,165–168另请参阅单个单元工具

Unit test tools, 165–168. See also by individual unit tools

行为驱动的开发工具,166–168

behavior-driven development tools, 166–168

列表,126-127

list of, 126–127

概述, 165

overview of, 165

单元测试

Unit testing

自动化, 282

automating, 282

BDD(行为驱动开发),165–168

BDD (Behavior-driven development), 165–168

定义,507

definition, 507

指标和,76

metrics and, 76

支持功能,5

supporting function of, 5

TDD(测试驱动开发)和,111

TDD (test-driven development) and, 111

面向技术的测试,120

technology-facing tests, 120

象限 1 测试的工具,126–127

tools for Quadrant 1 tests, 126–127

可用性测试,202–204

Usability testing, 202–204

检查竞争对手的申请,204

checking out applications of competitors, 204

导航和,204

navigation and, 204

概述, 202

overview of, 202

用户需求和角色测试,202-204

users needs and persona testing, 202–204

什么不应该自动化,285–286

what should not be automated, 285–286

用例,398

Use cases, 398

用户验收测试。参见 UAT(用户验收测试)

User acceptance testing. See UAT (user acceptance testing)

用户文档,207–208

User documentation, 207–208

用户界面 (UI)。另见GUI(图形用户界面)

User interface (UI). See also GUI (graphical user interface)

自动化策略和,293

automation strategy and, 293

建模和, 399

modeling and, 399

用户故事。故事

User story. See Story

用户故事卡。查看故事卡

User story card. See Story card

应用于敏捷软件开发的用户故事(Cohn),155

User Stories Applied for Agile Software Development (Cohn), 155

V

V

Vaage,卡罗尔,330

Vaage, Carol, 330

价值

Value

添加,31-33

adding, 31–33

交付给客户,22-23

delivering to customer, 22–23

专注于,341–342

focusing on, 341–342

测试人员添加,12

testers adding, 12

价值观,敏捷,3–4另请参阅原则,适用于敏捷测试人员

Values, agile, 3–4. See also Principles, for agile testers

变体、编码和测试以及410

Variations, coding and testing and, 410

速度

Velocity

自动化和, 255 , 484

automation and, 255, 484

职业倦怠率和,79

burnout rate and, 79

数据库影响,228

database impact on, 228

缺陷和,487

defects and, 487

定义,507

definition, 507

最大化,370

maximizing, 370

可持续的测试速度,41

sustainable pace of testing and, 41

花时间做正确的事,301

taking time to do it right, 301

技术债务和106 , 313 , 418 , 506

technical debt and, 106, 313, 418, 506

供应商

Vendors

自动化工具,315–316

automation tools, 315–316

捕捉回放工具,267

capture-playback tool, 267

集成开发环境,125

IDEs, 125

规划和342–344

planning and, 342–344

源代码控制工具,124

source code control tools, 124

142 , 349一起工作

working with, 142, 349

维拉根,皮埃尔,76 岁163 岁179 岁295 岁363 岁372 岁444 岁

Veragen, Pierre, 76, 163, 179, 295, 363, 372, 444

版本控制123–124,186另请参阅源代码控制

Version control, 123–124, 186. See also Source Code Control

观点。另见大图

Viewpoints. See also Big picture

在迭代规划期间考虑所有观点,385–389

considering all viewpoints during iteration planning, 385–389

收集有关需求的所有观点,374-375

gathering all viewpoints regarding requirements, 374–375

三的幂和,411

Power of Three and, 411

在引出需求时使用多种观点,137–138

using multiple viewpoints in eliciting requirement, 137–138

能见度,354–366

Visibility, 354–366

代码覆盖率,360–364

code coverage, 360–364

交流测试结果,357–358

communicating test results, 357–358

缺陷指标,364–366

defect metrics, 364–366

通过测试的次数,358–360

number of passing tests, 358–360

概述, 354

overview of, 354

发布指标,358

release metrics, 358

跟踪测试任务和状态,354–357

tracking test tasks and status, 354–357

视觉工作室,125

Visual Studio, 125

约翰·沃里斯,117 岁

Voris, John, 117

W

W

瀑布方法,开发

Waterfall approach, to development

与敏捷开发相比,12–13

agile development compared with, 12–13

“迷你瀑布”现象,46-47

ìmini-waterfallî phenomenon, 46–47

成功,112

successes of, 112

测试计划和346

test plans and, 346

Watir(Ruby中的Web应用程序测试),163、172–174、320另请参阅Ruby 与 Watir

Watir (Web Application Testing in Ruby), 163, 172–174, 320. See also Ruby with Watir

Web 服务描述语言 (WSDL),507

Web Services Description Language (WSDL), 507

网络服务测试

Web service testing

自动化, 282

automating, 282

概述, 207

overview of, 207

远程数据监控系统示例,247–248

Remote Data Monitoring system example, 247–248

工具,170–171

tools for, 170–171

网络负载,234

WebLoad, 234

惠兰,德克兰,321

Whelan, Declan, 321

白板

Whiteboards

示例驱动开发,379

example-driven development, 379

促进沟通,430

facilitating communication, 430

建模, 399

modeling, 399

规划图,371

planning diagram, 371

与程序员一起审查高级测试,400–401

reviewing high-level tests with programmers, 400–401

测试计划备选方案,353–354

test plan alternatives, 353–354

整个团队方法,325

Whole team approach, 325

的优点, 26

advantages of, 26

敏捷与传统开发,15–16

agile vs. traditional development, 15–16

自动化策略和,300-301

automation strategy and, 300–301

预算限制和55

budget limits and, 55

在工作中找到乐趣,31

finding enjoyment in work and, 31

关键成功因素, 482 , 491

key success factors, 482, 491

将测试人员与程序员配对,279

pairing testers with programmers, 279

共同责任和105–106

shared responsibility and, 105–106

团队建设,69

team building and, 69

团队结构和,59–62

team structure and, 59–62

测试自动化,270

to test automation, 270

测试管理和,322

test management and, 322

与传统跨职能团队相比,64

traditional cross-functional team compared with, 64

团队成员的价值和,70

value of team members and, 70

维基百科

Wiki

作为通讯工具,164

as communication tool, 164

示例的图形文档,398–399

graphical documentation of examples, 398–399

模型, 160 , 380

mockups, 160, 380

要求,402

requirements, 402

故事清单和,156

story checklists and, 156

测试用例,372

test cases, 372

可追溯性和88

traceability and, 88

威尔逊 - 威尔士,帕特里克,278

Wilson-Welsh, Patrick, 278

绿野仙踪测试,138-139

Wizard of Oz Testing, 138–139

工作流程图,398

Workflow diagrams, 398

有效地使用遗留代码 Feathers ),117,288

Working Effectively With Legacy Code (Feathers), 117, 288

工作量,393

Workload, 393

最坏情况136,334

Worst-case scenarios, 136, 334

编写测试,策略。参见Strategy,用于编写测试

Writing tests, strategy for. See Strategy, for writing tests

WSDL(网络服务描述语言),507

WSDL (Web Services Description Language), 507

X

X

XP(极限编程)

XP (Extreme Programming)

敏捷团队拥抱,10-11

agile team embracing, 10–11

勇气作为核心价值,25

courage as core value in, 25

xUnit, 126–127

xUnit, 126–127

Y

亚基奇,乔,316

Yakich, Joe, 316

Z

Z

零错误容忍79,418–419

Zero bug tolerance, 79, 418–419